ComputerVision

Normalize images with transform in pytorch dataloader

Training a deep learning model is a complicated task which not only needs to adjust the parameters such as learning, but also needs to process the training data. Normalized the data is one usefule approach and has been widely used for preparing dataset.

Normalized data allows faster convergence

In deep learning, the image data is normalized by the mean and stadard deviation. This helps get consistent results when applying a model to new images. Hence, the convergence of the model

Read More

Python

OpenSlide indroduction

OpenSlide

https://openslide.org/

OpenSlide is a C library that provides simple interfaces to process Whole-Slide-Images (also known as virtual slides). WSIs are very high resolution images used in digital pathology. These images usually have several gigabytes and can not be easily read with standard tools or libraries because the uncompressed images often occupy tens of gigabytes which consume huge of RAM. WSI is scanned with multi-resolution, while OpenSlide can support read a small amount of image data at the resolution closest to a specifical magnification level.

OpenSlide supports many popular WSI format:

  • Aperio (.svs, .tif)
  • Hamamatsu (.ndpi, .vms, .vmu)
  • Leica (.scn)
  • MIRAX (.mrxs)
  • Philips (.tiff)
  • Sakura (.svslide)
  • Trestle (.tif)
  • Ventana (.bif, .tif)
  • Generic tiled TIFF (.tif)

Although it is written by C, Python and Java bindings are also provided, and some other bindings are contributed in the Github such as Ruby, Rust.

Read More

Programming Language

Learn Prolog

Recently, I got in touch with the Prolog in my coursework (EEEM0005). It is not easy! Prolog is a very old programming language which I only see it in the TIOBE programming language rank (a boring index), but I was attracted by its simple structure and strange programming grammar at begining.

1
2
3
4
5
6
parent(pam, bob).
parent(tom, bob).
parent(tom, liz).
parent(bob, pat).
parent(bob, ann).
parent(pat, jim).

Above all six facts (also called clauses) represent a parent relationship:

Read More

Deep learning

Training Model on Nvidia DIGITS

About Nvidia DIGITS

GPU has high efficient on training model especially for Deep Neural Network. Honestly, training model is not an easy task that we need to prepare datasets, choose network and config lots of parameters. It is a big challenge for you don’t have much experience with programming and frameworks like Python, Py-torch. Nvidia DIGITS is a web platform which allows us train model with user friendly GUI without coding. DIGITS simplifies common deep learning tasks such as managing data, designing and training neural networks on multi-GPU systems, monitoring performance in real time with advanced visualizations, and selecting the best performing model from the results browser for deployment. DIGITS is completely interactive so that data scientists can focus on designing and training networks rather than programming and debugging.

In this article, I’ll use MNIST dataset and LeNet network to train a model which can classify numbers. Training process can be divided into three steps:

  1. Import Dataset
  2. Training
  3. Testing and analysing

Now we start. If you are confused with how to install DIGITS, please go to the document.

Read More