Skip to main content

linear-algebra


·         Eigen Value Decomposition [Lecture 6.3]

o    Works for square matrices

o    X = UU-1 will be the eigen value decomposition where U represent the eigen vectors of X and  represent diagonal matrix with eigen values

o    In case of orthogonal matrices, UTU = I, hence, X = UT is EVD of X

·         PCA - [YouTube Mitesh Khapra Unit 6]

o    Transforming the data into a new basis where the new axes have high variance and

o    How to do?

·         Normalize the data to have zero mean and unit variance

·         Calculate XTX and find its Eigen vectors. Stacking the Eigen vectors column-wise in matrix P would represent the transformation matrix for the new axis  (X^ = XP)

·         Estimate the points using only largest k Eigen vectors to get to an approximation  such that reconstruction error is minimized

·         Largest k Eigen values correspond to the axis which has highest variance for the data. Throwing away n-k eigen values will be fine

·         Example: Eigen Faces, Storage of faces (100x100=10k dimensions) using small dimensions (50-100)

·         SVD Singular value Decomposition [Lecture 6.8]

o    Works for Rectangular matrices - Even harder as they also transform the vector from Rn to Rm

o    X = UVT  where X , U, V .

o    Left Singular Matrix: U are eigen vectors of  XXT

o    Right Singular Matrix: V are eigen vectors of  XTX.

o       is a diagonal matrix with k singular values of X

o      Rank-k approximations of a matrix:  is the Rank-1 approximation of matrix X and is the Rank-k approximation of X. Where k largest singular values are taken

o    Singular values are absolute square root of eigen values

o    When does SVD behave the same as EVD?

·          Pseudo Inverse

o    In cases of rectangular matrices when inverse is not possible i.e. pseudo-inverse helps

o    Using SVD, pseudo inverse of a matrix can be easily found, See link below

o    https://inst.eecs.berkeley.edu/~ee127/sp21/livebook/def_pseudo_inv.html

 





Comments

Post a Comment

Popular posts from this blog

Most Important steps to win any Machine Learning Competition

Gaining experience from the blogs of all the winners of online competition, I could see what approaches they follow, their solutions etc. Some of the most important points that I try to follow when I see a new machine learning challenge are: 1. Understand : Try to understand the data thoroughly, what each feature means and how it it related to the target variable. Exploring the data using plots gives an idea about features.  2. Data cleaning : Imputation of missing values and data pre-processing.  3. Feature Engineering : The most important part is feature engineering. Good feature engineering can give you an edge over others in any competition. In fact, it's the most important and most difficult step to think of and comes only with experience and practice. 4. Splitting your data:  Creating a strategy how to split your data into training and validation set. A good validation set that matches your score on the leader board is very crucial to fine tune your code....
Post a Comment
Read more

Interesting Projects in Deep Learning

Deep Learning is the latest new area in machine learning which is proving handy to solve many interesting problems which were not possible to do with the standard ML algorithms before such as Language Translation, Text to speech, Speech Recognition etc. It has moved us one step closer to what we thought of as our objective. Self Driving cars are the latest invention of this field and is growing at a very fast rate with more number of developers opting to move to deep learning. Some of the most interesting projects in deep learning that helps you learn concepts as well as apply these concepts on real world data and complex use cases are: Tranfer Learning : To use a pretrained model to find objects in any image Style Transfer : Applying the style of one image onto the other image Digit Recognizer : Use CNN to find the handwritten digits with our own custom built CNN model MNIST data : Classify over 10 objects in the MNIST data TED Talks : Creating word embeddings in a dataset...
Post a Comment
Read more