TL;DR: Use this gist and you're all set! The remainder of this post is a step-by-step for a newbie to Deep Learning/AWS
The Deep Learning ecosystem has matured tremendously over the last few months or so. I've been playing around with some of these applications over the same time period as well, and it's amazing how much the field has moved ahead in such a short time. Today, terms like Word2Vec, CNN and LSTMs have became part of the lingo of nearly every researcher, hobbyist or otherwise.
And of course, AWS makes running deep-learning applications possible at almost no cost. Especially if you lack a good GPU on your personal system, AWS is a godsend. I've been working with some of these amazing frameworks so thought it might be good to automate the process of setting up a brand new EC2 instance with everything required to get up and running with deep learning. This includes:
1) Get yourself an AWS EC2 instance.
Go ahead and get yourself a new instance with an Ubuntu AMI (14.04/16.04 should be fine). Since we obviously we want to use the GPU instances, pick either g2.2xlarge or g2.8xlarge instance type. (Lost?)
2) SSH into the machine and either:
a) Download this zip file and extract it into the machine using:
i) deep_learning_bootstrap.sh
This is a shell script which installs all the required dependencies and libraries required.
ii) test_install.sh
This is a shell script which tests to ensure that the install is successful.
iii) theano_test.py
This is a small python script which tests to ensure Theano is using the GPU for it's computations.
3) Next up run the following to ensure the shell scripts are executable:
If a pink screen pops up mentioning "A new version of /boot/grub/menu.lst is available", choose "Keep local version" and select OK.
Once everything is done, you should see the following message:
And... that's it! You're all set to start developing your own deep learning applications on AWS EC2. Here's a set of examples you can get started with.
Happy coding! :)
Notes: Why both Theano and Tensorflow? Because choice. Honestly, thanks to Keras, using either as a backend is as simple as a string change. Also, most early to mid level posts in the field use one of the two with no strong monopoly, so it simply made sense to have both of them.
The Deep Learning ecosystem has matured tremendously over the last few months or so. I've been playing around with some of these applications over the same time period as well, and it's amazing how much the field has moved ahead in such a short time. Today, terms like Word2Vec, CNN and LSTMs have became part of the lingo of nearly every researcher, hobbyist or otherwise.
And of course, AWS makes running deep-learning applications possible at almost no cost. Especially if you lack a good GPU on your personal system, AWS is a godsend. I've been working with some of these amazing frameworks so thought it might be good to automate the process of setting up a brand new EC2 instance with everything required to get up and running with deep learning. This includes:
- TensorFlow
- Theano
- Keras
- CUDA
- CuDNN
- Misc. tools like tmux which are a godsend for remote development.
1) Get yourself an AWS EC2 instance.
Go ahead and get yourself a new instance with an Ubuntu AMI (14.04/16.04 should be fine). Since we obviously we want to use the GPU instances, pick either g2.2xlarge or g2.8xlarge instance type. (Lost?)
2) SSH into the machine and either:
a) Download this zip file and extract it into the machine using:
$ curl "http://link_to_zip" -o dl_setup.zipand unzip the file using
$ unzip dl_setup.zipb) Simply create the 3 files and copy the contents from the gist using your favorite editor. The 3 files are:
i) deep_learning_bootstrap.sh
This is a shell script which installs all the required dependencies and libraries required.
ii) test_install.sh
This is a shell script which tests to ensure that the install is successful.
iii) theano_test.py
This is a small python script which tests to ensure Theano is using the GPU for it's computations.
3) Next up run the following to ensure the shell scripts are executable:
chmod +x *.sh4) Run the bootstrap script using:
./deep_learning_bootstrap.shThis will take some time (~5-10 minutes) to download and install all the dependencies.
If a pink screen pops up mentioning "A new version of /boot/grub/menu.lst is available", choose "Keep local version" and select OK.
Once everything is done, you should see the following message:
Reboot System (sudo shutdown -r 0) and run ./test_install.sh5) Follow the instruction and run:
sudo shutdown -r 0Your SSH connection will disconnect while your machine reboots. Wait for ~30 seconds and SSH back into the machine. You should see the prompt changed to:
ubuntu@D33P_L34RN $6) Now run the test script to ensure everything has correctly installed:
./test_install.shJust look for the messages in green and ensure the output matches it.
And... that's it! You're all set to start developing your own deep learning applications on AWS EC2. Here's a set of examples you can get started with.
Happy coding! :)
Notes: Why both Theano and Tensorflow? Because choice. Honestly, thanks to Keras, using either as a backend is as simple as a string change. Also, most early to mid level posts in the field use one of the two with no strong monopoly, so it simply made sense to have both of them.
0 comments :
Post a Comment