Week 1 - Introduction to Deep Learning

Welcom

Hello and welcome. As you probably know, deep learning has already transformed traditional internet businesses like web search and advertising. 딥러닝은 이미 웹검색 및 광고와 같은 전통적인 인터넷 비즈니스를 변화 시켰다. But deep learning is also enabling brand new products and businesses and ways of helping people to be created. 그러나 딥러닝은 새로운 제품과 비즈니스 및 사람들이 만드는 방법을 가능하게 한다. Everything ranging from better healthcare, where deep learning is getting really good at reading X-ray images to delivering personalized education, to precision agriculture, to even self driving cars and many others. If you want to learn the tools of deep learning and be able to apply them to build these amazing things, I want to help you get there. 딥러닝 도구를 배우고 이 놀라운 것들을 만드는데 적용할 수 있다면 거기에 도달하는 것을 돕고 싶다.

When you finish the sequence of courses on Coursera, called the specialization, you will be able to put deep learning onto your resume` with confidence. Over the next decade, I think all of us have an opportunity to build an amazing world, amazing society, that is AI powers, and I hope that you will play a big role in the creation of this AI power society. So that, let's get started.

I think that AI is the new electricity. Starting about 100 years ago, the electrification of our society transformed every major industry, every ranging from transportation, manufacturing, to healthcare, to communications and many more. And today, we see a surprisingly clear path for AI to bring about an equally big transformation.

And of course, the part of AI that is rising rapidly and driving a lot of these developments, is deep learning. So today, deep learning is one of the most highly sought after skills and technology worlds. And through this course and a few causes after this one,

I want to help you to gain and master those skills. So here's what you learn in this sequence of courses also called a specialization on Coursera.

In the first course, you learn about the foundations of neural networks, you learn about neural networks and deep learning. 첫 번째 코스에서는 신경 네트워크의 기초에 대해 배우고, neural network 와 deep learning에 대해 배웁니다. This video that you're watching is part of this first course which last four weeks in total. And each of the five courses in the specialization will be about two to four weeks, with most of them actually shorter than four weeks.

But in this first course, you'll learn how to build a new network including a deep neural network and how to train it on data.

And at the end of this course, you'll be able to build a deep neural network to recognize, guess what? Cats. For some reason, there is a cat neem running around in deep learning. And so, following tradition in this first course, we'll build a cat recognizer. 첫번째 코스는 전통에 따라 고양이 인식기를 만들 것이다. Then in the second course, you learn about the practical aspects of deep learning. 두번째 코스에서는 deep learning의 실제적인 측면을 배우게 된다. So you learn, now that you've built in your network, how to actually get it to perform well. 네트워크가 구축되었으므로 이제 실제로 네트워크 성능을 향상시키는 방법을 배우게 된다. So you learn about hyperparameter tuning, regularization, how to diagnose price and variants and advance optimization algorithms like momentum armrest prop and the ad authorization algorithm. 따라서 하이퍼 매개변수 조정, 정규화, 가격 및 변형을 진단하는 방법 및 운동 armrest prop 및 광고 인증 알고리즘과 같은 고급 최적화 알고리즘에 대해 배운다.

Sometimes it seems like there's a lot of tuning, even some black magic and how you build a new network.

So the second course which is just three weeks, will demystify some of that black magic. In the third course which is just two weeks, you learn how to structure your machine learning project. It turns out that the strategy for building a machine learning system has changed in the era of deep learning. 딥러닝 시대에 기계학습 시스템을 구축하기 위한 전략이 바뀌였다.

So for example, the way you switch your data into train, development or dev also called holdout cross-validation sets and test sets, has changed in the era of deep learning.

So whether the new best practices are doing that and whether if you were training set and your test come from different distributions, that's happening a lot more in the era of deep learning.

So how do you deal with that? And if you've heard of end to end deep learning, you also learn more about that in this third course and see when you should use it and maybe when you shouldn't.

The material in this third course is relatively unique. I'm going to share of you a lot of the hard one lessons that I've learned, building and shipping, quite a lot of deep learning products. As far as I know, this is largely material that is not taught in most universities that have deep learning courses. But I really hope you to get your deep learning systems to work well. In the next course, we'll then talk about convolutional neural networks, often abbreviated CNNs. Convolutional networks or convolutional neural networks are often applied to images. So you learn how to build these models in course four. Finally, in course five, you learn sequence models and how to apply them to natural language processing and other problems. So sequence models includes models like recurrent neural networks abbreviated RNNs and LSTM models, sense for a long short term memory models. You'll learn what these terms mean in course five and be able to apply them to natural language processing problems. So you learn these models in course five and be able to apply them to sequence data. So for example, natural language is just a sequence of words, and you also understand how these models can be applied to speech recognition, or to music generation, and other problems. So through these courses, you'll learn the tools of deep learning, you'll be able to apply them to build amazing things, and I hope many of you through this will also be able to advance your career. So that, let's get started. Please go on to the next video where we'll talk about deep learning applied to supervise learning.

그렇다면 ML vs DL의 차이는 뭔가부터 알아 보고 가자!

https://brunch.co.kr/@itschloe1/8

What is a nenural network?

The term, Deep Learning, refers to training Neural Networks, sometimes very large Neural Networks. Deep Learning이라는 용어는 뉴럴네트워크를 교육하는 것을 말하며 때로는 매우 큰 신경 네트워크를 지칭한다. So what exactly is a Neural Network? 신경망은 정확히 무엇인가? In this video, let's try to give you some of the basic intuitions. 여기서는 기본적인 직관에 대해 설명해 보겠다. Let's start to the Housing Price Prediction example. Let's say you have a data sets with six houses, so you know the size of the houses in square feet or square meters and you know the price of the house and you want to fit a function to predict the price of the houses, the function of the size. So if you are familiar with linear regression you might say, well let's put a straight line to these data so and we get a straight line like that. But to be Pathans you might say well we know that prices can never be negative, right. So instead of the straight line fit which eventually will become negative, let's bend the curve here. So it just ends up zero here. So this thick blue line ends up being your function for predicting the price of the house as a function of this size. Whereas zero here and then there's a straight line fit to the right. 주택 가격은 음수가 나올 수 없으니 0으로 끝난다. 크기의 함수로 주택 가격을 예측하는 기능을 한다.

So you can think of this function that you've just fit the housing prices as a very simple neural network. 따라서 이 함수를 생각해 보면 방금 주택 가격을 아주 간단한 신경망으로 맞출 수 있다. It's almost as simple as possible neural network. Let me draw it here.

We have as the input to the neural network the size of a house which one we call x. 신경망에 대한 입력으로 우리는 x라고 부르는 집의 크기를 갖는다. It goes into this node, this little circle and then it outputs the price which we call y. 그것은 이 노드, 이 작은 동그라미에 들어가서 y라고 부르는 가격을 출력한다. So this little circle, which is a single neuron in a neural network, implements this function that we drew on the left.

And all the neuron does is it inputs the size, computes this linear function, takes a max of zero, and then outputs the estimated price. 모든 뉴련은 크기를 입력하고 이 선형 함수를 계산하고 최대값을 취한 다음 예상 가격을 출력한다.

And by the way in the neural network literature, you see this function a lot. This function which goes to zero sometimes and then it'll takes of as a straight line.

This function is called a ReLU function which stands for rectified linear units. So R-E-L-U. And rectify just means taking a max of 0 which is why you get a function shape like this.

You don't need to worry about ReLU units for now but it's just something you see again later in this course. So if this is a single neuron, neural network, really a tiny little neural network, a larger neural network is then formed by taking many of the single neurons and stacking them together. So, if you think of this neuron that's being like a single Lego brick, you then get a bigger neural network by stacking together many of these Lego bricks. Let's see an example.

Let’s say that instead of predicting the price of a house just from the size, you now have other features. You know other things about the host, such as the number of bedrooms, I should have wrote [INAUDIBLE] bedrooms, and you might think that one of the things that really affects the price of a house is family size, right? 집의 가격에 실제로 영향을 미치는 물건 중 하나가 가족의 크기라고 생각할 수 있다. So can this house fit your family of three, or family of four, or family of five? And it's really based on the size in square feet or square meters, and the number of bedrooms that determines whether or not a house can fit your family's family size. And then maybe you know the zip codes, in different countries it's called a postal code of a house. And the zip code maybe as a future to tells you, walkability? So is this neighborhood highly walkable? Thing just walks the grocery store? Walk the school? Do you need to drive? And some people prefer highly walkable neighborhoods. And then the zip code as well as the wealth maybe tells you, right. Certainly in the United States but some other countries as well. Tells you how good is the school quality. So each of these little circles I'm drawing, can be one of those ReLU, rectified linear units or some other slightly non linear function. So that based on the size and number of bedrooms, you can estimate the family size, their zip code, based on walkability, based on zip code and wealth can estimate the school quality. And then finally you might think that well the way people decide how much they're will to pay for a house, is they look at the things that really matter to them. In this case family size, walkability, and school quality and that helps you predict the price.

So in the example x is all of these four inputs. 예제에서 x는 이 네가지 입력 모두 이다. And y is the price you're trying to predict. y는 예측하려고 하는 가격 And so by stacking together a few of the single neurons or the simple predictors we have from the previous slide, we now have a slightly larger neural network.

How you manage neural network is that when you implement it, you need to give it just the input x and the output y for a number of examples in your training set and all this things in the middle, they will figure out by itself. 신경망을 갖게 되었을때 훈련세트에 있는 많은 예제와 중간에 있는 모든 것들에 대한 입력 x와 출력 y를 줄 필요가 있는가?

So what you actually implement is this. Where, here, you have a neural network with four inputs. So the input features might be the size, number of bedrooms, the zip code or postal code, and the wealth of the neighborhood. And so given these input features, the job of the neural network will be to predict the price y. 여기서 네 개의 입력을 가진 신경 네트워크가 있다. 입력된 기능은 크기, 침실 수 , 우편 번호 또는 이웃의 재산 등이 될 수 있다.

And notice also that each of these circle, these are called hidden units in the neural network, that each of them takes its inputs all four input features. So for example, rather than saying these first nodes represent family size and family size depends only on the features X1 and X2. Instead, we're going to say, well neural network, you decide whatever you want this known to be.

And we'll give you all four of the features to complete whatever you want. So we say that layers that this is input layer and this layer in the middle of the neural network are density connected.

Because every input feature is connected to every one of these circles in middle. And the remarkable thing about neural networks is that, given enough data about x and y, given enough training examples with both x and y, neural networks are remarkably good at figuring out functions that accurately map from x to y. 신경망에 대해 주목할만한 점은 x와 y에 대한 충분한 데이터가 주어지면 x와 y에 대한 충분한 훈련 예제가 주어지면 신경망은 x에서 y로 정확하게 매핑되는 함수를 계산할 때 탁월하다.

So, that's a basic neural network. In turns out that as you build out your own neural networks, you probably find them to be most useful, most powerful in supervised learning incentives, meaning that you're trying to take an input x and map it to some output y, like we just saw in the housing price prediction example. In the next video let's go over some more examples of supervised learning and some examples of where you might find your networks to be incredibly helpful for your applications as well. supervised learning에 대한 몇 가지 예와 네트워크가 응용 프로그램에 매우 도움이 될 수 있는 몇 가지 예를 살펴 보겠다.

Supervised Learning with Neural

There's been a lot of hype about neural networks. 신경망에 대해서는 많은 과장이 있다. And perhaps some of that hype is justified, given how well they're working.

But it turns out that so far, almost all the economic value created by neural networks has been through one type of machine learning, called supervised learning. 지금까지 신경 네트워크에 의해 생성된 거의 모든 경제적 가치는 supervised learning을 통해 이루어졌음이 밝혀졌다. Let's see what that means, and let's go over some examples. 그게 무슨 뜻인지 몇 가지 예를 들어보자. In supervised learning, you have some input x, and you want to learn a function mapping to some output y. So for example, just now we saw the housing price prediction application where you input some features of a home and try to output or estimate the price y. Here are some other examples that neural networks have been applied to very effectively.

Possibly the single most lucrative application of deep learning today is online advertising, 오늘날 가장 가장 탁월한 딥러닝은 온라인 광고이다. maybe not the most inspiring, but certainly very lucrative, in which, by inputting information about an ad to the website it's thinking of showing you, and some information about the user, 아마도 가장 고무적은 아니지만 확실히 수익성이 좋습니다. neural networks have gotten very good at predicting whether or not you click on an ad. And by showing you and showing users the ads that you are most likely to click on, this has been an incredibly lucrative application of neural networks at multiple companies. Because the ability to show you ads that you're more likely to click on has a direct impact on the bottom line of some of the very large online advertising companies. 광고에 대한 정보를 웹 사이트에 입력하여 사용자에 대한 정보를 표시하고 사용자에 대한 정보를 제공할 수 있다. 신경 네트워크는 광고를 클릭했는지 여부를 예측하는 데 매우 능숙하다.

그리고 당신을 보여주고 사용자가 가장 많이 클릭할 가능성이 있는 광고를 보여줌으로써 이것은 여러 회사에서 신경 네트워크의 엄청나게 유리한 응용프로그램이다. 클릭 가능성이 더 높은 광고를 게재할 수 있는 기능은 일부 대형 온라인 광고 회사의 수익에 직접적인 영향을 미칩니다.

Computer vision has also made huge strides in the last several years, mostly due to deep learning. 딥러닝을 통해서 컴퓨터 비전은 또한 지난 몇 년 동안 큰 발전을 이루었다. So you might input an image and want to output an index, say from 1 to 1,000 trying to tell you if this picture, it might be any one of, say a 1000 different images. So, you might us that for photo tagging.

I think the recent progress in speech recognition has also been very exciting, where you can now input an audio clip to a neural network, and have it output a text transcript.

오디오 클립을 신경망에 입력하고 텍스트 사본을 출력할 수 있다. Machine translation has also made huge strides thanks to deep learning where now you can have a neural network input an English sentence and directly output say, a Chinese sentence. ML은 DL덕분에 커다란 진보를 이루었다.이제는 신경망 입력을 통해 영어 문장을 입력하고 직접 중국어 문장을 출력할 수 있다.

And in autonomous driving, you might input an image, say a picture of what's in front of your car as well as some information from a radar, and based on that, maybe a neural network can be trained to tell you the position of the other cars on the road. 자율 주행에서는 이미지를 입력하고 레이더의 정보뿐만 아니라 차 앞의 그림을 말하고 그 정보에 따라 신경 네트워크를 훈련시켜 다른 사람의 위치를 알려줄 수 있다.

So this becomes a key component in autonomous driving systems. 딥러닝은 자율주행 시스템에서 핵심구성 요소가 된다. So a lot of the value creation through neural networks has been through cleverly selecting what should be x and what should be y for your particular problem, and then fitting this supervised learning component into often a bigger system such as an autonomous vehicle.

It turns out that slightly different types of neural networks are useful for different applications. 다른 유형의 신경망이 다른 응용 분야에 유용하다는 것이 밝혀졌다. For example, in the real estate application that we saw in the previous video, we use a universally standard neural network architecture, right? Maybe for real estate and online advertising might be a relatively standard neural network, like the one that we saw.

For image applications we'll often use convolution on neural networks, often abbreviated CNN. 이미지 애플리케이션의 경우 종종 신경 네트워크에서 회선(convolution)을 사용한다. CNN이라고 한다.

And for sequence data. So for example, audio has a temporal component, right? Audio is played out over time, so audio is most naturally represented as a one-dimensional time series or as a one-dimensional temporal sequence. 시퀸스 데이터. 예를 들어 오디오에는 시간적 요소가 있다. 오디오는 시간이 지남에 따라 재생되므로 오디오는 가장 자연스럽게 1차원 시계열 또는 1차원 시간 순서로 표현된다.

And so for sequence data, you often use an RNN, a recurrent neural network. 따라서 시퀸스 데이터의 경우 반복적인 신경망인 RNN을 사용하는 경우가 많다. Language, English and Chinese, the alphabets or the words come one at a time. So language is also most naturally represented as sequence data. 언어는 시퀸스 데이터로 가장 자연스럽게 표현된다. 따라서 RNN의 복잡한 버전이 이러한 응용 프로그램에 자주 사용된다.

And so more complex versions of RNNs are often used for these applications.

And then, for more complex applications, like autonomous driving, where you have an image, that might suggest more of a CNN convolution neural network structure and radar info which is something quite different. You might end up with a more custom, or some more complex, hybrid neural network architecture. 그리고 나서 이미지가 있는 자율 주행과 같은 더 복잡한 애플리케이션의 경우 CNN convolution 신경망 구조와 레이더 정보가 상당히 다른 것을 알 수 있다. 더 많은 사용자 정의 또는 좀 더 복잡한 하이브리드 신경망 아키텍처로 끝날수도 있다.

So, just to be a bit more concrete about what are the standard CNN and RNN architectures. 따라서 표준 CNN 및 RNN 아키텍처에 대해 좀 더 구체적으로 설명하세요. So in the literature you might have seen pictures like this. So that's a standard neural net.

You might have seen pictures like this. Well this is an example of a Convolutional Neural Network, and we'll see in a later course exactly what this picture means and how can you implement this.

But convolutional networks are often use for image data. And you might also have seen pictures like this. And you'll learn how to implement this in a later course.

Recurrent neural networks are very good for this type of one-dimensional sequence data that has maybe a temporal component. You might also have heard about applications of machine learning to both Structured Data and Unstructured Data. Here's what the terms mean. Structured Data means basically databases of data.

So, for example, in housing price prediction, you might have a database or the column that tells you the size and the number of bedrooms.

So, this is structured data, or in predicting whether or not a user will click on an ad, you might have information about the user, such as the age, some information about the ad, and then labels why that you're trying to predict.

So that's structured data, meaning that each of the features, such as size of the house, the number of bedrooms, or the age of a user, has a very well defined meaning.

In contrast, unstructured data refers to things like audio, raw audio, or images where you might want to recognize what's in the image or text. 반대로 구조화 되지 않은 데이터는 오디오, 원시 오디오 또는 이미지 또는 텍스트의 내용을 인식하고자하는 이미지와 같은 것을 말한다. Here the features might be the pixel values in an image or the individual words in a piece of text.

Historically, it has been much harder for computers to make sense of unstructured data compared to structured data. 컴퓨터가 구조화된 데이터에 비해 구조화 되지 않은 데이터를 이해하는 것이 훨씬 더 어렵다. And the fact the human race has evolved to be very good at understanding audio cues as well as images. 반면 사람은 구조화 되지 않은 데이터를 해석하는 데 정말로 능숙하다.

And then text was a more recent invention, but people are just really good at interpreting unstructured data.

And so one of the most exciting things about the rise of neural networks is that, thanks to deep learning, thanks to neural networks, computers are now much better at interpreting unstructured data as well compared to just a few years ago. 신경 네트워크의 부상에 대한 가장 흥미로운 사실 중 하나는 신경망 덕분에 컴퓨터가 단순한 몇 년 전에 비해 구조화되지 않은 데이터를 단순한 몇 년 전에 비해 구조화 되지 않은 데이터를 해석하는 데 훨씬 유리하다는 것이다.

And this creates opportunities for many new exciting applications that use speech recognition, image recognition, natural language processing on text, much more than was possible even just two or three years ago. 그리고 이것은 음석인식, 이미지 인식, 텍스트에 대한 자연어 처리, 2~3년만에 가능해 졌다.

I think because people have a natural empathy to understanding unstructured data, you might hear about neural network successes on unstructured data more in the media because it's just cool when the neural network recognizes a cat.

We all like that, and we all know what that means.

But it turns out that a lot of short term economic value that neural networks are creating has also been on structured data, such as much better advertising systems, much better profit recommendations, and just a much better ability to process the giant databases that many companies have to make accurate predictions from them. So in this course, a lot of the techniques we'll go over will apply to both structured data and to unstructured data. 그러나 신경망이 만들어 내는 단기간의 경제적 가치는 훨씬 우수한 광고 시스템, 더 나은 수익 추천, 거대 데이터베이스 처리 능력등 구조화 된 데이터에 기반하고 있다.

For the purposes of explaining the algorithms, we will draw a little bit more on examples that use unstructured data. But as you think through applications of neural networks within your own team I hope you find both uses for them in both structured and unstructured data.

So neural networks have transformed supervised learning and are creating tremendous economic value. 그래서 신경 네트워크는 supervised learning을 변형 시켰으며 엄청난 경제적 가치를 창출하고 있다. It turns out though, that the basic technical ideas behind neural networks have mostly been around, sometimes for many decades. 신경 네트워크의 기본 기술 아이디어는 대개 수십년 동안 주변에서 발생했다. So why is it, then, that they're only just now taking off and working so well? In the next video, we'll talk about why it's only quite recently that neural networks have become this incredibly powerful tool that you can use. 신경 네트워크가 당신이 사용할 수 있는 이 믿을 수 없는만큼 강력한 도구가 된 것은 최근에야 왜 그런지 이야기 할 것이다.

nowol79 / MOOC