There are various differentially private algorithms to support Differential Privacy.
Bounded Mean, Bounded Sum, Laplace Mechanism, Exponential Mechanism, Private Histogram, Secured Multi Party Computation (Secured MPC), Differentially Private SGD, etc.
Of these algorithms Secured MPC, is one that I have worked in multiple other projects within domain of cryptography and blockchain as well. For the interest of privacy within ML models we will go through Differentially Private SGD in detail as well. We will be using libraries to approach the implementation within our applications.
Lets go with few concepts on DP like Function Sensitivity, Privacy Loss, Privacy Budget, etc.
Function Sensitivity – signifies how sensitive a function is to changes. The max change in the output of function when single individual’s data in input dataset is altered.
Privacy Loss – extend to which addition or removal of single individual’s data in dataset can influence the output of the query or computation
Privacy Budget – ϵ | epsilon – total privacy spending allowed while maintaining acceptable privacy guarantees
Following Openmined from the very early days when it was founded, I was very fond of understanding how ML and blockchain comes together – Decentralized AI. Other companies like SingularityNet were there however Openmined was more interesting to me. Openmined combines Federated learning with Homomorphic Encryption HME and blockchain to enable collaborative model to implement machine learning application in decentralized fashion.
Libraries on DP:
- IBM Differential Privacy ( diffprivlib )
- Google Differential Privacy
- Openmined DP (pydp)
- Opendp dpcreator
We will go through multiple libraries for implementation of DP, starting with IBM Differential privacy – which I think is very straight forward and simple implementation.
IBM Differential Privacy
Installation
$ pip install diffprivlib