Wednesday, 20 September 2017

Deep Learning HOWTO with IBM Minsky Power8 server, Ubuntu 16.04, Caffe-NV and Kitti (Part 5).

This is the last part where we will optimize first the environment and then we will perform finally a test run of the Kitti test. :-)

Optimize the system

# Optimize the environment

root@minsky:~# apt install linux-tools-common linux-tools-4.4.0-62-generic linux-tools-generic

root@minsky:~# cpupower -c all frequency-set -g performance
Setting cpu: 0
Setting cpu: 1
Setting cpu: 8
Setting cpu: 9
.
.
.
Setting cpu: 113
Setting cpu: 120
Setting cpu: 121

root@minsky:~# ppc64_cpu –smt=2

root@minsky:~# nvidia-smi -pm ENABLED
Enabled persistence mode for GPU 0002:01:00.0.
Enabled persistence mode for GPU 000A:01:00.0.
All done.

root@minsky:~# nvidia-smi -ac 715,1480
Applications clocks set to "(MEM 715, SM 1480)" for GPU 0002:01:00.0
Applications clocks set to "(MEM 715, SM 1480)" for GPU 000A:01:00.0
All done.



# If you want you can unconfigure the Nvidia ECC memory capability too (should provide some extra advantage)

root@minsky:~# nvidia-smi -e 0 # 0 is a zero
root@minsky:~# reboot

Kitti test run

https://github.com/NVIDIA/DIGITS/blob/v4.0.0/examples/object-detection/README.md

Wget http://kitti.is.tue.mpg.de/kitti/data_object_image_2.zip
wget https://fredrikarneving.se/digits/data_object_label_2.zip --no-check-certificate
wget https://fredrikarneving.se/digits/devkit_object.zip --no-check-certificate
wget https://fredrikarneving.se/digits/caffe_nv_model.txt --no-check-certificate
wget https://fredrikarneving.se/digits/bvlc_googlenet.caffemodel --no-check-certificate


root@minsky:/sw/dw/data# cp ./devkit_object.zip $DIGITS_HOME/examples/object-detection/
root@minsky:/sw/dw/data# cp ./data_object_label_2.zip $DIGITS_HOME/examples/object-detection/
root@minsky:/sw/dw/data# cp data_object_image_2.zip $DIGITS_HOME/examples/object-detection/


root@minsky:/sw/dw/data# cd $DIGITS_HOME/examples/object-detection/
root@minsky:/sw/digits/examples/object-detection# ./prepare_kitti_data.py
Extracting zipfiles ...
Unzipping data_object_label_2.zip ...
Unzipping data_object_image_2.zip …
Unzipping devkit_object.zip ...
Calculating image to video mapping ...
Splitting images by video ...
Creating train/val split ...
Done.
root@minsky:/sw/digits/examples/object-detection#

# Follow the instructions in the URL until you get this web page

# After some minutes it will have finished
# Run the test as the URL suggests, using: Dataset = "Kitti default" Epochs = 100 Subtract mean = "None" Batch size = 16 Solver type = "Adam" Base Learning rate = 0.0001 Custom Network = https://raw.githubusercontent.com/NVIDIA/caffe/caffe-0.15/examples/kitti/detectnet_network.prototxt Pretrained model = https://github.com/BVLC/caffe/tree/rc3/models/bvlc_googlenet # You should get something similar to this: # AS reference a similar Supermicro Intel based server with 2xP100 GPUs runs this test in 300 minutes.

Interesting links


http://developer.download.nvidia.com/compute/cuda/repos/

18 comments:

markson said...

What sort of equipment and programming will be required to set up this information stockroom?
data science course in pune

seoexpert said...

Nice Post...I have learn some new information.thanks for sharing.
ExcelR data analytics course in Pune | business analytics course | data scientist course in Pune

seoexpert said...

Such a very useful article. I have learn some new information.thanks for sharing.
data scientist course in mumbai

Manikanta said...

Such a very useful article. Very interesting to read this article. I have learn some new information.thanks for sharing. ExcelR

ExcelR Pune said...

Very nice blog here and thanks for post it.. Keep blogging...
ExcelR data science training

ravali said...

After reading your article I was amazed. I know that you explain it very well. And I hope that other readers will also experience how I feel after reading your article.
ExcelR data analytics courses

Jagna Co Kalani said...

Great Article
Data Mining Projects


Python Training in Chennai

Project Centers in Chennai

Python Training in Chennai

Excelrsolutions said...

This is a wonderful article, Given so much info in it, These type of articles keeps the users interest in the website, and keep on sharing more ... good luck... data science course

Data Science Course said...

Really awesome blog!!! I finally found a great post here.I really enjoyed reading this article. Nice article on data science . Thanks for sharing your innovative ideas to our vision. Your writing style is simply awesome with useful information. Very informative, Excellent work! I will get back here.
Data Science Course Training in Bangalore

Unknown said...

The information provided on the site is informative. Looking forward more such blogs. Thanks for sharing .
Artificial Inteligence course in Varanasi
AI Course in Varanasi

Author said...

Happy to see so many people enjoying this article :-) I promise to try to post something else soon.

priyanka said...

wonderful article. Very interesting to read this article.I would like to thank you for the efforts you had made for writing this awesome article. This article resolved my all queries.
Data science Interview Questions

ravali said...

This is a wonderful article, Given so much info in it, These type of articles keeps the users interest in the website, and keep on sharing more ... good luck.

data science course

priyash said...

Very interesting to read this article.I would like to thank you for the efforts you had made for writing this awesome article. This article inspired me to read more. keep it up.
Correlation vs Covariance

ravali said...

I like viewing web sites which comprehend the price of delivering the excellent useful resource free of charge. I truly adored reading your posting. Thank you!

Simple Linear Regression

ek said...

wow, great, I was wondering how to cure acne naturally. and found your site by google, learned a lot, now I am a bit clear. I’ve bookmarked your site. keep us updated.
<a href="https://www.excelr.com/business-analytics-training-in-pune/”> Business Analytics Courses</a>
I have express a few of the articles on your website now, and I really like your style of blogging. I added it to my favorite’s blog site list and will be checking back soon…

priyanka said...

Very interesting to read this article.I would like to thank you for the efforts you had made for writing this awesome article. This article inspired me to read more. keep it up.
Correlation vs Covariance
Simple linear regression
data science interview questions

priyash said...

Very interesting to read this article.I would like to thank you for the efforts you had made for writing this awesome article. This article inspired me to read more. keep it up.
Correlation vs Covariance
Simple linear regression
data science interview questions

Holidays are for the summer