Markdown formula not working

[Infinity4B]

I've noticed that many formula are not working in https://github.com/purepisces/Wenqing-Machine_Learning_Blog/blob/main/Deep-Learning-Concepts/Basic-Neural-Network-Concepts/Activation-Functions/activation-functions.md, like: https://github.com/purepisces/Wenqing-Machine_Learning_Blog/blob/f17fbe801106919df9ad1dcdaf0dd51f4d38a793/Deep-Learning-Concepts/Basic-Neural-Network-Concepts/Activation-Functions/activation-functions.md?plain=1#L293 and https://github.com/purepisces/Wenqing-Machine_Learning_Blog/blob/f17fbe801106919df9ad1dcdaf0dd51f4d38a793/Deep-Learning-Concepts/Basic-Neural-Network-Concepts/Activation-Functions/activation-functions.md?plain=1#L319

Moreover, there are excess $ sign in: https://github.com/purepisces/Wenqing-Machine_Learning_Blog/blob/f17fbe801106919df9ad1dcdaf0dd51f4d38a793/Deep-Learning-Concepts/Basic-Neural-Network-Concepts/Activation-Functions/activation-functions.md?plain=1#L235 and https://github.com/purepisces/Wenqing-Machine_Learning_Blog/blob/f17fbe801106919df9ad1dcdaf0dd51f4d38a793/Deep-Learning-Concepts/Basic-Neural-Network-Concepts/Activation-Functions/activation-functions.md?plain=1#L250

These are small mistakes, and I hope these questions won't bother you.

[purepisces]

I've noticed that many formula are not working in https://github.com/purepisces/Wenqing-Machine_Learning_Blog/blob/main/Deep-Learning-Concepts/Basic-Neural-Network-Concepts/Activation-Functions/activation-functions.md, like:

https://github.com/purepisces/Wenqing-Machine_Learning_Blog/blob/f17fbe801106919df9ad1dcdaf0dd51f4d38a793/Deep-Learning-Concepts/Basic-Neural-Network-Concepts/Activation-Functions/activation-functions.md?plain=1#L293

and https://github.com/purepisces/Wenqing-Machine_Learning_Blog/blob/f17fbe801106919df9ad1dcdaf0dd51f4d38a793/Deep-Learning-Concepts/Basic-Neural-Network-Concepts/Activation-Functions/activation-functions.md?plain=1#L319

Moreover, there are excess $ sign in:

https://github.com/purepisces/Wenqing-Machine_Learning_Blog/blob/f17fbe801106919df9ad1dcdaf0dd51f4d38a793/Deep-Learning-Concepts/Basic-Neural-Network-Concepts/Activation-Functions/activation-functions.md?plain=1#L235

and https://github.com/purepisces/Wenqing-Machine_Learning_Blog/blob/f17fbe801106919df9ad1dcdaf0dd51f4d38a793/Deep-Learning-Concepts/Basic-Neural-Network-Concepts/Activation-Functions/activation-functions.md?plain=1#L250

These are small mistakes, and I hope these questions won't bother you.

Thank you so much for your feedback! I’ve made all the suggested changes. I'm thrilled to know that my machine learning blog is being noticed and appreciated. Please feel free to reach out or open issues anytime—I'd love to hear more from you! 😉

[purepisces]

I've noticed that many formula are not working in https://github.com/purepisces/Wenqing-Machine_Learning_Blog/blob/main/Deep-Learning-Concepts/Basic-Neural-Network-Concepts/Activation-Functions/activation-functions.md, like:

https://github.com/purepisces/Wenqing-Machine_Learning_Blog/blob/f17fbe801106919df9ad1dcdaf0dd51f4d38a793/Deep-Learning-Concepts/Basic-Neural-Network-Concepts/Activation-Functions/activation-functions.md?plain=1#L293

and https://github.com/purepisces/Wenqing-Machine_Learning_Blog/blob/f17fbe801106919df9ad1dcdaf0dd51f4d38a793/Deep-Learning-Concepts/Basic-Neural-Network-Concepts/Activation-Functions/activation-functions.md?plain=1#L319

Moreover, there are excess $ sign in:

https://github.com/purepisces/Wenqing-Machine_Learning_Blog/blob/f17fbe801106919df9ad1dcdaf0dd51f4d38a793/Deep-Learning-Concepts/Basic-Neural-Network-Concepts/Activation-Functions/activation-functions.md?plain=1#L235

and https://github.com/purepisces/Wenqing-Machine_Learning_Blog/blob/f17fbe801106919df9ad1dcdaf0dd51f4d38a793/Deep-Learning-Concepts/Basic-Neural-Network-Concepts/Activation-Functions/activation-functions.md?plain=1#L250

These are small mistakes, and I hope these questions won't bother you.

If you'd like, you could star my repository to make it easier for you to find. Also, I'm curious—how did you hear about my repo? Thank you so much :)

[Infinity4B]

Seems like i forgot to star your repository, i've starred now. I find your repo by searching for sigmoid function on Bing, and see your post on Zhihu: https://zhuanlan.zhihu.com/p/681952195. The tutorial is very nice, but the post on Zhihu is not comprehensive as here, so here i am.

[purepisces]

Seems like i forgot to star your repository, i've starred now. I find your repo by searching for sigmoid function on Bing, and see your post on Zhihu: https://zhuanlan.zhihu.com/p/681952195. The tutorial is very nice, but the post on Zhihu is not comprehensive as here, so here i am.

Oh I see! yes, the markdown format is not working smoothly on Zhihu, so I stopped update on this platform for a long time. You can just check my github repo and current progress https://github.com/purepisces/Wenqing-Machine_Learning_Blog/progress.txt. If you have other questions, you're very welcome to send message to me.

[Infinity4B]

Seems like i forgot to star your repository, i've starred now. I find your repo by searching for sigmoid function on Bing, and see your post on Zhihu: https://zhuanlan.zhihu.com/p/681952195. The tutorial is very nice, but the post on Zhihu is not comprehensive as here, so here i am.

Oh I see! yes, the markdown format is not working smoothly on Zhihu, so I stopped update on this platform for a long time. You can just check my github repo and current progress https://github.com/purepisces/Wenqing-Machine_Learning_Blog/progress.txt. If you have other questions, you're very welcome to send message to me.

Sure, I have found missing $ sign here: https://github.com/purepisces/Wenqing-Machine_Learning_Blog/blob/16e1fb95836800709c876520b19d52fe79502325/Deep-Learning-Concepts/Basic-Neural-Network-Concepts/Neural-Network-Layers/Linear-Layer.md?plain=1#L245 and https://github.com/purepisces/Wenqing-Machine_Learning_Blog/blob/16e1fb95836800709c876520b19d52fe79502325/Deep-Learning-Concepts/Basic-Neural-Network-Concepts/Neural-Network-Layers/Linear-Layer.md?plain=1#L247 and https://github.com/purepisces/Wenqing-Machine_Learning_Blog/blob/16e1fb95836800709c876520b19d52fe79502325/Deep-Learning-Concepts/Basic-Neural-Network-Concepts/Neural-Network-Layers/Linear-Layer.md?plain=1#L249

[purepisces]

Excellent, I've modified, thank you! and

Seems like i forgot to star your repository, i've starred now. I find your repo by searching for sigmoid function on Bing, and see your post on Zhihu: https://zhuanlan.zhihu.com/p/681952195. The tutorial is very nice, but the post on Zhihu is not comprehensive as here, so here i am.

Oh I see! yes, the markdown format is not working smoothly on Zhihu, so I stopped update on this platform for a long time. You can just check my github repo and current progress https://github.com/purepisces/Wenqing-Machine_Learning_Blog/progress.txt. If you have other questions, you're very welcome to send message to me.

Sure, I have found missing $ sign here:

https://github.com/purepisces/Wenqing-Machine_Learning_Blog/blob/16e1fb95836800709c876520b19d52fe79502325/Deep-Learning-Concepts/Basic-Neural-Network-Concepts/Neural-Network-Layers/Linear-Layer.md?plain=1#L245

and https://github.com/purepisces/Wenqing-Machine_Learning_Blog/blob/16e1fb95836800709c876520b19d52fe79502325/Deep-Learning-Concepts/Basic-Neural-Network-Concepts/Neural-Network-Layers/Linear-Layer.md?plain=1#L247

and https://github.com/purepisces/Wenqing-Machine_Learning_Blog/blob/16e1fb95836800709c876520b19d52fe79502325/Deep-Learning-Concepts/Basic-Neural-Network-Concepts/Neural-Network-Layers/Linear-Layer.md?plain=1#L249

Excellent, I've modified, thank you! And as pointed in progress.txt, 22.Linear-Layer.md✅ need further modification, I need to modify the logic and structure in this file, and will update it later for a better version. I will try to modify it this weekend for you to have a better understanding of linear layer. :D

[Infinity4B]

That's very nice of you! Look forward to it ;>

[Infinity4B]

What does the $\iota_N$ and $\iota_C$ mean here? https://github.com/purepisces/Wenqing-Machine_Learning_Blog/blob/e36a8d3dd2998bcfcbad9a445bd5acba9aafb4b8/Deep-Learning-Concepts/Basic-Neural-Network-Concepts/Loss-Functions/Cross_Entropy_Loss.md?plain=1#L46 Moreover, it there any wrong here? https://github.com/purepisces/Wenqing-Machine_Learning_Blog/blob/e36a8d3dd2998bcfcbad9a445bd5acba9aafb4b8/Deep-Learning-Concepts/Basic-Neural-Network-Concepts/Loss-Functions/Cross_Entropy_Loss.md?plain=1#L108 I think it should be:

$$\begin{align} \frac{\partial \log(\sigma(A{ik}))}{\partial A{ic}} &= \frac{\partial \log(\sigma(A{ik}))}{\partial \sigma(A{ik})} \cdot \frac{\partial \sigma(A{ik})}{\partial A{ic}} \ &= \frac{1}{\sigma(A{ik})} \cdot -\sigma(A{ik}) \cdot \sigma(A{ic}) \ &= -\sigma(A{ic}) \end{align}$$

[Infinity4B]

Also, i find a mistake here: https://github.com/purepisces/Wenqing-Machine_Learning_Blog/blob/e36a8d3dd2998bcfcbad9a445bd5acba9aafb4b8/Deep-Learning-Concepts/Basic-Neural-Network-Concepts/Loss-Functions/Cross_Entropy_Loss.md?plain=1#L231 It should be:

$$-\begin{bmatrix} y_1+y_2+y_3 - a_1(y_1 + y_2 + y_3) & y_2 - a_2(y_1 + y_2 + y_3) & y_3 - a_3(y_1 + y_2 + y_3) \end{bmatrix}$$

[purepisces]

Wenqing-Machine_Learning_Blog/Deep-Learning-Concepts/Basic-Neural-Network-Concepts/Loss-Functions/Cross_Entropy_Loss.md

Hi [Infinity4B], I'm so sorry that I'm too busy these 2 weeks, will reply on this weekend with detail explanation, also, keep posting your questions here. If you find there is a problem or mistake, just keep posting. Thank you so much! :)

[Infinity4B]

I've found another missing $ sign here: https://github.com/purepisces/Wenqing-Machine_Learning_Blog/blob/4a006315afdf9e013a30850e78784457885a6591/Foundational-Concepts-in-Machine-Learning/Optimization-Algorithms/gradient-descent.md?plain=1#L22

I've already read what i want to know. There are some concepts i want to know that are sadly missing(like Regularization). I especially like some of your chapters with code implementation, which helps me a lot. It is a very nice blog, and i hope you can keep updating! :))))

[purepisces]

What does the ι N and ι C mean here?

https://github.com/purepisces/Wenqing-Machine_Learning_Blog/blob/e36a8d3dd2998bcfcbad9a445bd5acba9aafb4b8/Deep-Learning-Concepts/Basic-Neural-Network-Concepts/Loss-Functions/Cross_Entropy_Loss.md?plain=1#L46

Moreover, it there any wrong here? https://github.com/purepisces/Wenqing-Machine_Learning_Blog/blob/e36a8d3dd2998bcfcbad9a445bd5acba9aafb4b8/Deep-Learning-Concepts/Basic-Neural-Network-Concepts/Loss-Functions/Cross_Entropy_Loss.md?plain=1#L108

I think it should be: ∂ l o g ⁡ ( σ ( A i k ) ) ∂ A i c = ∂ l o g ⁡ ( σ ( A i k ) ) ∂ σ ( A i k ) ⋅ ∂ σ ( A i k ) ∂ A i c = 1 σ ( A i k ) ⋅ − σ ( A i k ) ⋅ σ ( A i c ) = − σ ( A i c )

Sorry for the late reply, I'm too busy these days.

Answer to first question: $\iota_N$ is a column vector of size $N$, which means it has $N$ rows and 1 column, also every element in this vector is 1. $\iota_C$ is a column vector of size $C$ where all elements are 1. I use it in this formula $\text{crossentropy} = H(A, Y) = (-Y \odot \log(\sigma(A))) \cdot \mathbf{\iota}_C$. The shape of $(-Y \odot \log(\sigma(A)))$ is $(N \times C)$, so I multiply it with $\mathbf{\iota}_C$, resulting in a vector of size $(N \times 1)$, where each element corresponds to the cross-entropy loss for a specific sample. Then to transform it into a scalar, we can either use the sum or mean of all cross-entropy, as pointed in markdown here: https://github.com/purepisces/Wenqing-Machine_Learning_Blog/blob/e36a8d3dd2998bcfcbad9a445bd5acba9aafb4b8/Deep-Learning-Concepts/Basic-Neural-Network-Concepts/Loss-Functions/Cross_Entropy_Loss.md?plain=1#L56

Answer to second question: Excellent! you are correct, thank you so much for poining out 🎃

[purepisces]

Also, i find a mistake here:

https://github.com/purepisces/Wenqing-Machine_Learning_Blog/blob/e36a8d3dd2998bcfcbad9a445bd5acba9aafb4b8/Deep-Learning-Concepts/Basic-Neural-Network-Concepts/Loss-Functions/Cross_Entropy_Loss.md?plain=1#L231

It should be: − [ y 1 + y 2 + y 3 − a 1 ( y 1 + y 2 + y 3 ) y 2 − a 2 ( y 1 + y 2 + y 3 ) y 3 − a 3 ( y 1 + y 2 + y 3 ) ]

For this one, the result is the first row multiply with the first column, which is:

$$\begin{align} \frac{y_1}{a_1} \cdot a_1(1 - a_1) + \frac{y_2}{a_2} \cdot (-a_2 a_1) + \frac{y_3}{a_3} \cdot (-a_3 a_1) \ =y_1 (1 - a_1) + y_2 (-a_1) + y_3 (-a_1) \ =y_1 - a_1 y_1 - a_1 y_2 - a_1 y_3 \ =y_1 - a_1 (y_1 + y_2 + y_3) \end{align}$$

[purepisces]

I've found another missing $ sign here:

https://github.com/purepisces/Wenqing-Machine_Learning_Blog/blob/4a006315afdf9e013a30850e78784457885a6591/Foundational-Concepts-in-Machine-Learning/Optimization-Algorithms/gradient-descent.md?plain=1#L22

I've already read what i want to know. There are some concepts i want to know that are sadly missing(like Regularization). I especially like some of your chapters with code implementation, which helps me a lot. It is a very nice blog, and i hope you can keep updating! :))))

Wow, thank you so much! Actually, these days I'm woking on my deep learning system repo, hopefully when I finished this repo, it will help you to have a better understanding of deep learning. I will integrate the contents to my machine learning blog. For example, it includes auto differentiation, GPU acceleration, CNN RNN LSTM's implementation. It is like a self_defined pytorch library. I think the code implementation part is valuable, I'm so glad you feel the same way! :D

[purepisces]

I've found another missing $ sign here:

https://github.com/purepisces/Wenqing-Machine_Learning_Blog/blob/4a006315afdf9e013a30850e78784457885a6591/Foundational-Concepts-in-Machine-Learning/Optimization-Algorithms/gradient-descent.md?plain=1#L22

I've already read what i want to know. There are some concepts i want to know that are sadly missing(like Regularization). I especially like some of your chapters with code implementation, which helps me a lot. It is a very nice blog, and i hope you can keep updating! :))))

I will make a part about Regularization when I finished the deep learning system repo, and will let you know. Regularization is really an important part, thank you for reminding me of it:D

[purepisces]

I've found another missing $ sign here:

https://github.com/purepisces/Wenqing-Machine_Learning_Blog/blob/4a006315afdf9e013a30850e78784457885a6591/Foundational-Concepts-in-Machine-Learning/Optimization-Algorithms/gradient-descent.md?plain=1#L22

I've already read what i want to know. There are some concepts i want to know that are sadly missing(like Regularization). I especially like some of your chapters with code implementation, which helps me a lot. It is a very nice blog, and i hope you can keep updating! :))))

Also, you give me so much motivation for updating the blog, thank you so much, just keep posting your question. I'm happy to receive the feedback, and thank you for reading it so carefully :)

[Infinity4B]

Great! I'm very excited to learn about deep learning system and the code implementation of classical models! It's the most comprehensive blog I've ever found, and it's my pleasure to make it prefect. Looking forward to your update :P

[purepisces]

I've found another missing $ sign here:

https://github.com/purepisces/Wenqing-Machine_Learning_Blog/blob/4a006315afdf9e013a30850e78784457885a6591/Foundational-Concepts-in-Machine-Learning/Optimization-Algorithms/gradient-descent.md?plain=1#L22

I've already read what i want to know. There are some concepts i want to know that are sadly missing(like Regularization). I especially like some of your chapters with code implementation, which helps me a lot. It is a very nice blog, and i hope you can keep updating! :))))

I updated the regularization part: https://github.com/purepisces/Wenqing-Machine_Learning_Blog/blob/main/Deep-Learning-Concepts/Basic-Neural-Network-Concepts/Regularization/regularization-overview.md, also, highly recommend reading https://github.com/purepisces/Wenqing-Machine_Learning_Blog/blob/main/Foundational-Concepts-in-Machine-Learning/Classification-Methods/linear-classification.md to have a better understanding of it. So far, I just wrote the overview of regularization and in the future I will add specific markdown file about L1 and L2 regularization part. Hope that helps:D

[Infinity4B]

I've found another missing $ sign here: https://github.com/purepisces/Wenqing-Machine_Learning_Blog/blob/4a006315afdf9e013a30850e78784457885a6591/Foundational-Concepts-in-Machine-Learning/Optimization-Algorithms/gradient-descent.md?plain=1#L22

I've already read what i want to know. There are some concepts i want to know that are sadly missing(like Regularization). I especially like some of your chapters with code implementation, which helps me a lot. It is a very nice blog, and i hope you can keep updating! :))))

I updated the regularization part: https://github.com/purepisces/Wenqing-Machine_Learning_Blog/blob/main/Deep-Learning-Concepts/Basic-Neural-Network-Concepts/Regularization/regularization-overview.md, also, highly recommend reading https://github.com/purepisces/Wenqing-Machine_Learning_Blog/blob/main/Foundational-Concepts-in-Machine-Learning/Classification-Methods/linear-classification.md to have a better understanding of it. So far, I just wrote the overview of regularization and in the future I will add specific markdown file about L1 and L2 regularization part. Hope that helps:D

Thx! I've learnt a lot about the SVM in this blog, and the theory of High-Dimensional Points is interesting. I've also found disfunctional formula here: https://github.com/purepisces/Wenqing-Machine_Learning_Blog/blob/dc0c35eb610e3fb1be64b105299e9c8313c2f76c/Foundational-Concepts-in-Machine-Learning/Classification-Methods/linear-classification.md?plain=1#L162 and https://github.com/purepisces/Wenqing-Machine_Learning_Blog/blob/dc0c35eb610e3fb1be64b105299e9c8313c2f76c/Foundational-Concepts-in-Machine-Learning/Classification-Methods/linear-classification.md?plain=1#L166

[purepisces]

I've found another missing $ sign here: https://github.com/purepisces/Wenqing-Machine_Learning_Blog/blob/4a006315afdf9e013a30850e78784457885a6591/Foundational-Concepts-in-Machine-Learning/Optimization-Algorithms/gradient-descent.md?plain=1#L22

I've already read what i want to know. There are some concepts i want to know that are sadly missing(like Regularization). I especially like some of your chapters with code implementation, which helps me a lot. It is a very nice blog, and i hope you can keep updating! :))))

I updated the regularization part: https://github.com/purepisces/Wenqing-Machine_Learning_Blog/blob/main/Deep-Learning-Concepts/Basic-Neural-Network-Concepts/Regularization/regularization-overview.md, also, highly recommend reading https://github.com/purepisces/Wenqing-Machine_Learning_Blog/blob/main/Foundational-Concepts-in-Machine-Learning/Classification-Methods/linear-classification.md to have a better understanding of it. So far, I just wrote the overview of regularization and in the future I will add specific markdown file about L1 and L2 regularization part. Hope that helps:D

Thx! I've learnt a lot about the SVM in this blog, and the theory of High-Dimensional Points is interesting. I've also found disfunctional formula here:

https://github.com/purepisces/Wenqing-Machine_Learning_Blog/blob/dc0c35eb610e3fb1be64b105299e9c8313c2f76c/Foundational-Concepts-in-Machine-Learning/Classification-Methods/linear-classification.md?plain=1#L162

and https://github.com/purepisces/Wenqing-Machine_Learning_Blog/blob/dc0c35eb610e3fb1be64b105299e9c8313c2f76c/Foundational-Concepts-in-Machine-Learning/Classification-Methods/linear-classification.md?plain=1#L166

Thank you for pointing out, I used images instead. Welcome to check it out:D If you have other machine learning material want to learn, please let me know. It's my motivation to update this blog:)