Release notes

1.7.0

Main changes

Support creating many backends.
Support global shared DSP job queue provided by underlying ElcoreCL 1.3.
Update ONNX converter version to 2.3.1.
Support more ONNX operators and Keras layers.
Support 6D tensors.
Add example of YOLOv8 usage.
Add notes about MCom-03 ALT Linux packages in Getting started section.
Bug fixes.

ElcoreNN CPU API

Support many backends: InitBackend now returns ENNBackendId. LoadModel accepts ENNBackendId argument.
InitBackend without specifying devices list creates a backend in which cores for execution a model are dynamically selected by scheduler and global DSP queue. For more details see DSP assignment.
Add ReleaseBackend and ReleaseModel functions.

ONNX converter

Update to 2.3.1 version.
URL is changed to https://box.elvees.com/index.php/s/elcorenn-converters?path=%2FONNX%20converter

ONNX operators

Support:

Abs.
ArgMax.
Audiospectrogram.
AveragePool for 1D.
Conv 2D with groups.
Div with broadcasting.
Floor.
LSTM for float16.
MaxPool for 1D.
Mel-frequency cepstral coefficients (MFCCs).
Mod.
Neg.
Pooling layers: add ceil_mode.
PRelu.
ReduceL2.
ReduceMax.
ReduceSum.
ScatterND.
Split.

Keras layers

Support AveragePooling1D.
Support groups argument for Conv2D.

elcorenn-examples

sample-yolov5 example has been renamed to sample-yolo. Positional argument was added to choose type of YOLO: YOLOv5s, YOLOv8n. YOLO models are taken from Ultralytics YOLOv8.2.2.

Bug fixes

Fix bug when some layers as Conv, Dense work incorrectly after Reshape layer.
Fix bug when GetInputShape suddenly returns wrong value.
Fix bug with memory leaking in InvokeModel function.

1.6.0

Main changes

ElcoreNN DSP library optimizations.
Support Tanh and InstanceNormalization ONNX operators.

ElcoreNN DSP

Increase performance of following layers: Concat, Reshape, Split.

1.5.0

Main changes

New ElcoreNN CPU API functions.
ElcoreNN DSP library optimizations.
Update onnx-elcorenn to v1.3.3.
Support new ONNX operators and Keras layers.
elcorenn-examples: Installation path has been changed.

ElcoreNN CPU API

InvokeModel:

Supported different types of model input data.

Added multiple outputs support for input dmabuf (for models with one input).

dmabuf_input_user_pointer_output.cpp

#include "elcorenn/elcorenn.h"

int main() {
  // Init ElcoreNN resources
  InitBackend();
  // Load model from files
  auto id = LoadModel("path-to-model-json-file", "path-to-weights-bin-file");

  // Read output layers shapes
  int outputs_number = GetOutputsNumber(id);
  std::vector<std::vector<int>> outputs_shapes;
  for (int i = 0; i < outputs_number; i++){
    uint32_t shape_arr[MAX_NDIM_VALUE + 1];
    GetOutputShape(id, i, shape_arr);
    std::vector<int> shape(shape_arr[0]);
    for (int j = 0; j < shape_arr[0]; j++) {
      shape[j] = shape_arr[j + 1];
    }
    outputs_shapes.push_back(shape);
  }

  std::vector<int> buffers;
  uint32_t batch_size;
  // Create and fill or use already existing dmabufs to push it back to
  // buffers vector. Number of dmabufs in buffers should be equal to
  // batch_size.
  // ...
  // batch_size is a batch dimension value in input_data, output_data

  // Allocate memory for outputs data
  std::vector<std::vector<float>> outputs_data;
  for (int i = 0; i < outputs_number; i++){
    auto shape = outputs_shapes[i];
    int size = 1;
    for (auto dim_size: shape){
      size = size * dim_size;
    }
    std::vector<float> output_data(size * batch_size);
    outputs_data.push_back(output_data);
  }

  // Prediction
  float* outputs[outputs_number];
  for (int j = 0; j < outputs_number; j++) {
    outputs[j] = outputs_data[j].data();
  }
  InvokeModel(id, buffers.data(), outputs, batch_size);
}

Added support for input dmabuf and output dmabuf (for models with one input and one output).

dmabuf_input_dmabuf_output.cpp

#include "elcorenn/elcorenn.h"

int main() {
  // Init ElcoreNN resources
  InitBackend();
  // Load model from files
  auto id = LoadModel("path-to-model-json-file", "path-to-weights-bin-file");

  std::vector<int> input_buffers;
  std::vector<int> output_buffers;
  uint32_t batch_size;
  // Create or use already existing dmabufs to push it back to buffers
  // vector. Number of dmabufs in buffers should be equal to batch_size.
  // ...
  // batch_size is a batch dimension value in input_data, output_data

  // Prediction
  InvokeModel(id, input_buffers.data(), output_buffers.data(), batch_size);
}

InitBackend: Add argument dsp_heap to set dsp heap size.
LoadModel: Add argument optimization to set data type of computation.

ElcoreNN DSP

Add fp16 support for Pad layer.
Increase performance of sigmoid by using precalculated table values.
Increase performance by fusing layers:

into:

X * Sigmoid(X) function uses precalculated table values.

onnx-elcorenn

Update to v1.3.3 version.
Save converter version to model json file.
Support python3.8.
Support Dropout and Cast layers.
Bug fixes.

ONNX Operators

AveragePool: 2D, dilations=1.
Clip.
Conv: support dimension=3 for fp32.
ConvTranspose: support dimension=1.
LSTM: reference version (fp32 only).
MatMul.

Keras layers

Conv1DTranspose: reference version (fp32).
Conv3D: reference version (fp32).
LSTM: reference version (fp32).

elcorenn-examples

Installation path has been changed to /usr/bin.
prepare-examples-env.sh has been renamed to elcorenn-examples-get-data.sh.

Examples look for models relative to $HOME directory:

Go into $HOME directory.
Run elcorenn-examples-get-data.sh script to get models.

1.4.0

Main changes

Add new layers support.
Add YOLOv5 example.
Update onnx-elcorenn converter to 1.3.0 version.
Update ElcoreNN CPU API.
Create web storage of converters and elcorenn-examples-data releases.

ElcoreNN CPU

Support for models with multiple inputs and outputs.
Support dma_buf as input of model.
Add functions to get name and shapes of model inputs or outputs.

ElcoreNN DSP

Add Div, Erf, Conv1D, Sqrt layers.

elcorenn-examples

Add new example of YOLOv5s usage.
Update archive with video and models.

onnx-elcorenn

Update onnx-elcorenn to 1.3.0 version.
Support models converted from Tensorflow to ONNX.
Support attribute sizes for Resize operator. It represents target size of the output tensor.

Compatibility: elcorenn == v1.4.0