Label propagation on the Manifold for one-shot learning: propagate labels on the manifold(connected samples) and instead of marking the nearest one (nearest neighbor) to mark it as blue mark it as red where the nearest propagated label is red: link

One shot learning Andrew Ng    https://www.youtube.com/watch?v=jyPYdnEqCAk

Triplet Loss, for training choose samples where d(A,P) ~ d(A, N)

==========> Today I learned that
- one shot learning can be done via using embedding of classes taken from e.g. word2vec instead of multiclass and label propagation in manifold
  multi-million face recognition can be done using facenet and triplet loss as done in Google and taught by Ng in a single-shot learning lecture through siamese network

-----------------

m: # training examples

x: "input" variable/features

y: "output" variable/target variable

(x, y) one training example

ith training example (ith row in table) (x⁽ⁱ⁾, y⁽ⁱ⁾)

training set -> learning algorthm -> hypothesis h: input new living area, output estimated price

For this example h = θ + 

x1 size (ft2) x2 @ bedrooms

linear hypothesis class    h(x) = h_θ(x) = θ₀ + θ₁*x₁ + θ₂*x₂    to be concise in notation x₀ = 1, n # features    h(x) = Σ_{(i=0 to n)} θ_i x_i = θ^T x, θ's are parameters and the purpose is to learn right parameters

min_θ 1/2 Σ_{(i=0 to m)} (h_θ(x⁽ⁱ⁾) - y⁽ⁱ⁾)²     minimize the error for the training set  that we already know their output value, 1/2 is for conveniece which will reduce our future formulas

j(θ) = 1/2 Σ_{(i=0 to m)} (h_θ(x⁽ⁱ⁾) - y⁽ⁱ⁾)² 

min_θ j(θ)

start with some θ (θ = 0 vector) keep changing θ to reduce j(θ)

-------------

Gradient decent: θi := θi - α  ∂/∂θi  j(θ)   (:= replace the value)

 ∂/∂θi  j(θ) = Σ_{(k=0 to m)} (h_θ(x^(k)) - y^(k))² . x_i^(k)

α is the parameter of the learning algorithm called learning rate. controls how large a step you take

Repeat until convergence the following steps

θi := θi - α Σ_{(k=0 to m)} (h_θ(x^(k)) - y^(k))² . x_i^(k) = ∂/∂θ_i j(θ)       least squares is bell shaped has no local minima

as you approach local minimum gradient converges to zero

after multiple iterations of the gradient decent (Taking steps started from an arbitrary initial point) we get the least square fit. ==> linear regression, linear fit. The gradient of a function(derivative) gives the direction of the steepest descent. ==> Batch Gradient Descent : on every step look at all the samples. 

_{Stochastic Gradient Descent:} 

Repeat{ for k = 1 to m{ θi := θi - α hθ(x(k)) - y(k))2 . xi(k)  //update all parameters for all values of i } } to update parameters: only look at one training example each time. (the derivative w.r.t. just the first training example)

    

---

LECTURE 3

-----------------

Linear Regression

Locally Weighted Regression

Probabilistic Interpretation

Logistic Regression (First classification algorithm)

Digression (Preceptron)

Newton's method

Locally weighted regression LWR (Loess/Lowes)

LR (linear regression) fit θ to minimize least squares and return θ^T x

min_θ Σ_{(i=0 to m)} ( y⁽ⁱ⁾ - θ^T x⁽ⁱ⁾)² 

Locally Weighted Linear Regression: look at point x, lokk at datapoints, only takes data that aare in the little vicinity of x: where I want to evaluate the hypothesis, in its vicinity take a linear regression and where we hit that line fro our x is going to be our hypothesis value (predicted value).

 |x⁽ⁱ⁾-x| is small, then exp(0) is one from the formula then w⁽ⁱ⁾ ~ 1

 |x⁽ⁱ⁾-x| is large, then w⁽ⁱ⁾ ~ 0 :pay more attention to the points more close by more accurately