Getting Started

Tiny ONNC is an MLIR-based compiler exporting deep neural networks (DNN) into function calls to the ARM CMSIS-NN library. MLIR is a high-quality compiler framework addressing software fragmentation issues. By supporting variant Intermediate Representations in a single infrastructure, compilers can transform variant input languages into a common output form. Tiny ONNC leverages the unique power of MLIR to support rich neural network frameworks, including PyTorch, Open Neural Network Exchange Format (ONNX), Tensorflow, TensorflowLite, and even TVM relay. Tiny ONNC transforms all the input DNN formats into a function composed of a series of function calls to the ARM CMSIS-NN library. One fits all, MLIR makes it possible.

In this tutorial, we introduce you to a new web service - ONNC bench - to operate Tiny ONNC. Let’s build a new model today!

Install ONNC Bench

You can install onnc-bench by the popular pip. Simply type this:

$ pip3 install onnc-bench

To check your installation, use pip show.

$ pip3 show onnc-bench
Name: onnc-bench
Version: 1.3.1
Summary: ONNC-bench is a Python wrapper of ONNC
Author: The Skymizer Team
License: Apache License 2.0
Location: /usr/local/lib/python3.9/site-packages
Requires: sentry-sdk, click, wget, loguru, onnx, read-only-attributes, numpy, jinja2, requests

Create a Workspace

A workspace is a directory containing neural network projects. We leave useful tools in workspaces. You can think it as a working desktop with tool kits.

Creating a workspace is easy. onnc-create helps you to create a new workspace.

For example, if we want to create a new workspace named ws, we can use below instruction.

$ onnc-create ws
SUCCESS [/home/user1/ws] Create directory structure,
SUCCESS [/home/user1/ws] Generate scripts
SUCCESS [/home/user1/ws] Done

onnc-create will creates a new directory and leave some useful toolkits in the directory.

$ cd ws
$ ls
LICENSE.txt build-project create-project deploy-project

Get a Free API Key

ONNC bench is a software as a service (SaaS). To activate it, please register an account and get free API key first.


The registration website is here. After registration, we will send a confirmation mail to your email account. Please confirm your account and then visit the website again.

Join a Free Program

You can join our free program here. We will send a API key to your email account.

Fill you API Key

onnc-login program activates your API key. It’s an interactve script. Please key in your API key.

$ onnc-login
Please register your account and get API key from
Entry API-key: _

Great! Now we can start to create a project in the workspace.

Start from a Project Template

In the workspace, there are some useful scripts. We leave the scripts in workspace in case you want to customize it.

Create a Project

create-project script will create a new project. You can use -t option to select a template.

$ ./create-project -t vww -o my_prj
Cloning into 'my_prj'...
remote: Enumerating objects: 11, done.
remote: Counting objects: 100% (11/11), done.
remote: Compressing objects: 100% (8/8), done.
remote: Total 11 (delta 0), reused 8 (delta 0), pack-reused 0
Unpacking objects: 100% (11/11), done.
SUCCESS [Download vww example]
SUCCESS [Create `my_prj`]
$ ls ./my_prj     model.h5        requirement.txt samples.npy

To see all available projects, use -l option to get the list.

Build a Project

To transform a neural network model into a C source code, you can use build-project script.

The script will upload the model to ONNC bench SaaS, do quantization and compilation on the cloud, and then download the compiled results.

$ ./build-project -t ./my_prj -b NUMAKER_IOT_M487
SUCCESS [Upload model]
SUCCESS [Upload calibration samples]
INFO {'ram': 2490, 'rom': 101970}
SUCCESS [Calibration and compilation]
SUCCESS [Download deployment]

Extract Results from a Project

deploy-project script extracts compilation results from a project.

$ ./deploy-project -t ./my_prj -o output
SUCCESS [Copy results]

-o option specifies the output folder.

$ ls
LICENSE.txt build-project create-project deploy-project my_prj output

Review the Compilation Results

The compilation results include the transformed C source code and an estimation of memory usage.

$ ls ./output
compile_report.json src
$ cat compile_report.json

compile_report.json is a report of compiler.

    "ram": 2490,
    "rom": 101970

The produced source code are put in src/ folder.

$ ls ./output/src
cortexm_main.cpp    cortexm_runtime.cpp cortexm_weight.h    inference.h
cortexm_main.h      cortexm_runtime.h   inference.cpp       main.cpp