The Lexer Artifact and Flush Function

Lexer provides a function to export important data about your model to an artifact with the suffix .lexer.

The .lexer artifact will contain three files explained below:

• .toml file
• .onnx file
• .csv file

🚧

Underlying .lexer artifact

At time of this writing, the .lexer artifact here is essentially a zip folder. This may not be the case in the future though as we may explore more robust formats that fit our tooling needs.

This folder can be generated using a function named flush() that is made available through The Lexer Context.

Here's an example of using the flush() function:

lexer_context = LexerContext()
lexer_context.flush(name="your_artifact_name", target_directory=".")

When the Lexer flush() function is called, it will generate a zip folder called your_artifact_name.lexer, where your_artifact_name is from the name parameter passed into the above function above.

The artifact's output location will be wherever target_directory is set to ("." will place the .lexer file in the same directory as the file or script which called the function).

📘

Opening .lexer artifacts

You may use a CLI tool like unzip to open the .lexer artifact and see its contents:

Bash

unzip benchmark.lexer

Alternatively, you may find it easier to just rename it to .zip and open it via a GUI.

Lexer Artifact Metadata TOML File

The .toml file provides metadata about your application, such as when it was executed and what versions of Lexer and Python were used. It is for informational purposes and is not meant to be edited.

Example benchmark.toml File

# This is an autogenerated TOML Lexer artifact file.
# Please do not modify it directly.

title = "benchmark.lexer"

created_at = "2023-02-01T00:12:56.840550+00:00" # in UTC
updated_at = "2023-02-01T00:12:56.840550+00:00" # in UTC

[environment]
lexer_sdk_version = "0.0.1"
python_version = "3.10.0"
system = "Darwin"
release = "22.2.0"

[benchmark.config]
[benchmark.config.export_config]
batch_size = 1
enable_export_validation = true
enable_onnx_checks = true

[benchmark.config.benchmark_config]
enable_onnxruntime = true
enable_pytorch = true
num_iterations = 5

[benchmark.config.output_config]
enable_csv = true
enable_stdout = true

[benchmark.config.decorator_config]
input_names = ["input"]
output_names = ["output"]

[benchmark.config.output_config_internal]
onnx_output_filename = ""

[export.config]
[export.config.export_config]
batch_size = 1
enable_export_validation = true
enable_onnx_checks = true

[export.config.benchmark_config]
enable_onnxruntime = true
enable_pytorch = true
num_iterations = 1000000 # number of iterations to run all benchmarks

[export.config.output_config]
enable_csv = true
enable_stdout = true

[export.config.decorator_config]
input_names = ["input"]
output_names = ["output"]

[export.config.output_config_internal]
onnx_output_filename = ""

Timestamps are in UTC.

If you're unfamiliar with the TOML format, you can learn more at the official TOML website.

ONNX File

The Open Neural Network Exchange (ONNX), as described by the organization, is:

an open format built to represent machine learning models. ONNX defines a common set of operators - the building blocks of machine learning and deep learning models - and a common file format to enable AI developers to use models with a variety of frameworks, tools, runtimes, and compilers.

If the configuration enable_onnx_checks is set to true, then the benchmark.lexer folder will contain an .onnx file named after the class which you've wrapped with the The Lexer Decorator.

For example, if your class is called SuperResolutionNet, then this file will be named SuperResolutionNet.onnx.

If you're unfamiliar with the enable_onnx_checks setting, check out our guide to Configuring Lexer.

If you want to learn more about the ONNX file format, the ONNX organization (as part of The Linux Foundation) provides an introductory guide.

CSV File

The CSV file provides a human-readable output of the benchmark() function results. For a list of the benchmark data available, see The Lexer Decorator benchmarks.

The CSV file name is based on the class which you've wrapped with the [The Lexer Decorator](https://docs.lexer.ai/docs/the-lexer-decorator, so if your class is called SuperResolutionNet then the file will be named SuperResolutionNet.csv.

Looking to the Future

The Lexer artifact schema will play a crucial role in defining all generated assets when Lexer is used. This includes, but is not limited to, different machine learning model flavors - ONNX, other graph IR formats, TensorRT engines with various combinations of engine precisions and compute capability versions, and many more!

Lineage of these Lexer artifacts are also incredibly important in an ML lifecycle, so we have plans to provide ways to do automatic tagging via auto generated signatures and/or custom user tags.

Stay tuned for more!