Wednesday 20 September 2017

Deep Learning HOWTO with IBM Minsky Power8 server, Ubuntu 16.04, Caffe-NV and Kitti (Part 5).

This is the last part where we will optimize first the environment and then we will perform finally a test run of the Kitti test. :-)

Optimize the system

# Optimize the environment

root@minsky:~# apt install linux-tools-common linux-tools-4.4.0-62-generic linux-tools-generic

root@minsky:~# cpupower -c all frequency-set -g performance
Setting cpu: 0
Setting cpu: 1
Setting cpu: 8
Setting cpu: 9
.
.
.
Setting cpu: 113
Setting cpu: 120
Setting cpu: 121

root@minsky:~# ppc64_cpu –smt=2

root@minsky:~# nvidia-smi -pm ENABLED
Enabled persistence mode for GPU 0002:01:00.0.
Enabled persistence mode for GPU 000A:01:00.0.
All done.

root@minsky:~# nvidia-smi -ac 715,1480
Applications clocks set to "(MEM 715, SM 1480)" for GPU 0002:01:00.0
Applications clocks set to "(MEM 715, SM 1480)" for GPU 000A:01:00.0
All done.



# If you want you can unconfigure the Nvidia ECC memory capability too (should provide some extra advantage)

root@minsky:~# nvidia-smi -e 0 # 0 is a zero
root@minsky:~# reboot

Kitti test run

https://github.com/NVIDIA/DIGITS/blob/v4.0.0/examples/object-detection/README.md

Wget http://kitti.is.tue.mpg.de/kitti/data_object_image_2.zip
wget https://fredrikarneving.se/digits/data_object_label_2.zip --no-check-certificate
wget https://fredrikarneving.se/digits/devkit_object.zip --no-check-certificate
wget https://fredrikarneving.se/digits/caffe_nv_model.txt --no-check-certificate
wget https://fredrikarneving.se/digits/bvlc_googlenet.caffemodel --no-check-certificate


root@minsky:/sw/dw/data# cp ./devkit_object.zip $DIGITS_HOME/examples/object-detection/
root@minsky:/sw/dw/data# cp ./data_object_label_2.zip $DIGITS_HOME/examples/object-detection/
root@minsky:/sw/dw/data# cp data_object_image_2.zip $DIGITS_HOME/examples/object-detection/


root@minsky:/sw/dw/data# cd $DIGITS_HOME/examples/object-detection/
root@minsky:/sw/digits/examples/object-detection# ./prepare_kitti_data.py
Extracting zipfiles ...
Unzipping data_object_label_2.zip ...
Unzipping data_object_image_2.zip …
Unzipping devkit_object.zip ...
Calculating image to video mapping ...
Splitting images by video ...
Creating train/val split ...
Done.
root@minsky:/sw/digits/examples/object-detection#

# Follow the instructions in the URL until you get this web page

# After some minutes it will have finished
# Run the test as the URL suggests, using: Dataset = "Kitti default" Epochs = 100 Subtract mean = "None" Batch size = 16 Solver type = "Adam" Base Learning rate = 0.0001 Custom Network = https://raw.githubusercontent.com/NVIDIA/caffe/caffe-0.15/examples/kitti/detectnet_network.prototxt Pretrained model = https://github.com/BVLC/caffe/tree/rc3/models/bvlc_googlenet # You should get something similar to this: # AS reference a similar Supermicro Intel based server with 2xP100 GPUs runs this test in 300 minutes.

Interesting links


http://developer.download.nvidia.com/compute/cuda/repos/

8 comments:

markson said...

What sort of equipment and programming will be required to set up this information stockroom?
data science course in pune

Manikanta said...

Such a very useful article. Very interesting to read this article. I have learn some new information.thanks for sharing. ExcelR

Data Science Course said...

Really awesome blog!!! I finally found a great post here.I really enjoyed reading this article. Nice article on data science . Thanks for sharing your innovative ideas to our vision. Your writing style is simply awesome with useful information. Very informative, Excellent work! I will get back here.
Data Science Course Training in Bangalore

Unknown said...

The information provided on the site is informative. Looking forward more such blogs. Thanks for sharing .
Artificial Inteligence course in Varanasi
AI Course in Varanasi

Author said...

Happy to see so many people enjoying this article :-) I promise to try to post something else soon.

ek said...

wow, great, I was wondering how to cure acne naturally. and found your site by google, learned a lot, now I am a bit clear. I’ve bookmarked your site. keep us updated.
<a href="https://www.excelr.com/business-analytics-training-in-pune/”> Business Analytics Courses</a>
I have express a few of the articles on your website now, and I really like your style of blogging. I added it to my favorite’s blog site list and will be checking back soon…

priyanka said...

Very interesting to read this article.I would like to thank you for the efforts you had made for writing this awesome article. This article inspired me to read more. keep it up.
Correlation vs Covariance
Simple linear regression
data science interview questions

priyash said...

Very interesting to read this article.I would like to thank you for the efforts you had made for writing this awesome article. This article inspired me to read more. keep it up.
Correlation vs Covariance
Simple linear regression
data science interview questions

Holidays are for the summer