Intro to classification with MobileNet

Are you ready to discover classification with MobileNet? In this module, you will create your own image classification application! A few years ago, being able to automatically detect the content of a picture was science fiction, now you can do it easily on a smartphone!

Quiz

Watch the video above first, then answer the quiz to make sure you understand the main notions. Some questions may need to look up elsewhere through a quick Internet search!

This quiz is mandatory. You can answer this quiz as many times as you want, only your best score will be taken into account. Simply reload the page to get a new quiz.

The due date has expired. You can no longer answer this quiz.

Assignment

Make it happen!

This assignment is mandatory. If you update your work but the link doesn't change, you don't need to re-submit it.

Tools

p5.js Web Editoris a web editor for p5.js, a JavaScript library with the goal of making coding accessible to artists, designers, educators, and beginners.

TensorFlow.js is a machine learning framework developped by Google. MobileNet, the model we are using for this project is one of its available pre-trained models.

MobileNet classify images with labels from the ImageNet database. MobileNets are small, low-latency, low-power models parameterized to meet the resource constraints of a variety of use cases.

Project

Log in on p5.js web editor with your GitHub account and create a copy of this project

If you don't have a GitHub account, now is the time to sign up!
Use MobileNet to create an application that detects automatically at least two objects of your choice from the classes available on MobileNet.

You can find the full list of the classes that our pre-trained model can detect on its GitHub repository.

Describe an use case associated with your application in your webpage. We expect at least an explanation on who would be your users and how they would benefit from your application.

Describe an use case associated with your application in your webpage. For example, who would be your users and how they would benefit from your application.

Submit

The due date has expired. You can no longer submit your work.

Before submitting, make sure you check all of the criteria below.

Your sketch uses MobileNet to automatically detect at least two objects of your choice
Your webpage fully describes an use case associated with these detections
You did this work by yourself, and it’s not someone else’s

Going further

machine learning subfields — Classification is a subfield of Supervised learning, itself a subfield of Machine Learning

Definitions

Machine Learning is a subfield of Artificial Intelligence, it's a research domain that develops algorithms, called models, which "learns" from datas instead of classical algorithm which implements a set of predefined rules. The more (quality) datas you give to the algorithm, to more accurate it gets (to a certain extend).
Supervised Learning is a subfield of Machine Learning, it's a type of models where your training datasets includes both inputs and outputs. It means that you have a first set of datas (the training dataset) where you know the expected output(s) and you want the model to be able to extrapolate and predict a new output from data it has never encountered. The opposite is Unsupervised Learning, when you're trying to make sense of a dataset without knowing the expected output (for example finding similarities in a client database).
Classification is a subfield of Supervised Learning, where the expected output is discrete (think different categories, or yes/no). The opposite, when the output is continuous (a number), is called regression.
MobileNets are a series of Convolutional Neural Networks, that are made to do image recognition tasks on low-latency and low-power application (embedded or mobile for example)
ImageNetis an image database of nearly 15 million images ordered by categories.. that's why we can use them to train classification models! It's also the name of different machine learning competitions (see this one for example)

Keep in mind that with MobileNet you are only scratching the surface of what can be done with classification. As soon as you see a model that outputs a category, a yes/no answer or an action to do it's a classification!

Max AI: a robot performing a classification task... sorting trash for recycling!

Tools

TensorFlow, is an open-source Python framework developped by Google, originally developped for doing complex calculations, it quickly became one of the most used tools for machine learning.
Tensorflow.js, is its Javascript implementation. You just learned how to use MobileNet, one of its pre-trained models.
ml5.js, is a Javascript framework developed by teachers from NYU. Built on top of Tensorflow.js it enables you to quickly use pre-trained model in your browser.
Runway ML, is a desktop app that makes it easy to try a lot a machine learning models.

Resources

TensorFlow Playground, let you with a neural network directly in a browser. You will learn how machine learning models really "learns" from datas and what is an hyperparameter.
2D Visualisation of a CNN, trying to recognise handwritten digits.
iNaturalist, an example of a crowdsourcing website for wildlife conservation

**Jller**
Artistic implementation of classification

**AIpod**
Musical setting from research papers