모델 만들기

1. List

• List of DL-tricks

2. Paper

Efficient Processing of Deep Neural Networks: A Tutorial and Survey

3. Article (Post, blog, etc.)

• 기초용어/개념

• How to debug neural networks. Manual.

• Types of Optimization Algorithms used in Neural Networks and Ways to Optimize Gradient Descent

• 머신러닝 시스템 프로세스와 아키텍쳐

• Applying deep learning to real-world problems

• Learning I: the value of pre-training
• Learning II: caveats of real-world label distributions

• Learning III: understanding black box models

• An end to end implementation of a Machine Learning pipeline: github

• 37 Reasons why your Neural Network is not working: 원문, 번역

• My Neural Network isn't working! What should I do?

• Hyperopt tutorial for Optimizing Neural Networks’ Hyperparameters

• 14 DESIGN PATTERNS TO IMPROVE YOUR CONVOLUTIONAL NEURAL NETWORKS>/del>: 추천

• 10 steps to bootstrap your machine learning project: (part 1), (part 2)]

• Improve your neural networks: (Part 1), (Part 1한글)

• Must Know Tips/Tricks in Deep Neural Networks (by Vincent Granville)

• Must Know Tips/Tricks in Deep Neural Networks (by Xiu-Shen Wei)

• Neural Network Tuning with TensorFlow

• Machine Learning Pipeline: 조대협, 머신러닝 시스템 프로세스와 아키텍쳐

• How to Prevent Overfitting

• Model evaluation, selection and algorithm selection

• Train your deep model faster and sharper — two novel techniques

• Activation functions and it’s types-Which is better?

Am I overfitting?

• Very high accuracy on the training dataset (eg: 0.99)

• Poor accuracy on the test data set (0.85)

• 머신러닝 모델 개발 삽질 경험기: 조대협

• 학습 조기종료 시키기: 조기 종료하는 시점

• Pytorch 를 이용한 fine-tuning 및 feature extraction 코드

[추천]전처리

• cifarnet_preprocessing.py
• inception_preprocessing.py
• lenet_preprocessing.py
• preprocessing_factory.py

• vgg_preprocessing.py

• Where do I call the BatchNormalization function in Keras?

• How To Improve Deep Learning Performance

• Catastrophic forgetting 결함에 대한 문제를 극복하는 방법: 논문, 코드

Here are some resources for tips and tricks for training neural networks. (Resource 1, Resource 2, Resource 3)

• Limitation / Further Readings: catastrophic forgetting

QnA

• Why does learning curve of my model shows large variance in training error? How to fix it?

• Live loss plots inside Jupyter Notebook for Keras

• How big should batch size and number of epochs be when fitting a model in Keras?

• Hyperparameter search benchmark?

• How to design deep convolutional neural networks?: 결론은 정해진 룰 없음. 경험적으로 반복 수행으로 찾아 내는것임

3. Tutorial (Series, )

4. Youtube

6. Material (Pdf, ppt)

7. Implementation (Project)

8. Research Group / Conference

• github:DEvol - Deep Neural Network Evolution

DeepLearning Project Workflow라고 읽고 Bias-Variance Tradeoff에 가까운데 가끔 비슷한 질문이 올라오는 것 같아서 올립니다.

모델 성능이 원하는대로 나오지 않을때

1) 데이터를 더 모아야 하는지 2) 더 큰 모델을 만들어야 할 지

High bias, High Variance 체크를 통해 간단히 판단하는 법입니다. http://fbsight.com/t/deeplearning-project-workflow-bias-variance-tradeoff/78035

Some important parameters to look out for while optimizing neural networks are:
- Type of architecture
- Number of Layers
- Number of Neurons in a layer
- Regularization parameters
- Learning Rate
- Type of optimization / backpropagation technique to use
- Dropout rate
- Weight sharing
- etc.
- etc for CNN : convolutional filter size, pooling value, etc.

Tools

-Xcessiv: 웹에서 모델과 데이터를 지정하고 하이퍼파라미터 탐색을 위한 조건을 걸어서 자동으로 튜닝해주는 코드

-

나만의 모델을 만들어 보자

• 참고 [Youtube], [ppt]

• [추천]Use Keras Deep Learning Models with Scikit-Learn in Python: Keras와 Scikit-Learn을 이용한 성능향상

• Evaluate the Performance Of Deep Learning Models in Kerasn

• How to Grid Search Hyperparameters for Deep Learning Models in Python With Keras