The ASUS UGen300 USB AI Accelerator integrates a formidable 40 TOPS Hailo-10H AI processor alongside 8GB of LPDDR4 memory, optimizing functionalities for large language models (LLMs), vision language models (VLMs), and various AI vision applications—all while drawing less than 2.5 Watts.
In certain respects, this hardware echoes the capabilities of the Raspberry Pi AI HAT+ 2; however, it is presented in a sleek dongle form factor (measuring 105 x 50 x 18mm) and is equipped with a 10 Gbps USB 3.2 Gen1 USB-C Type-C interface, as opposed to the 5 or 8 GT/s PCIe connectivity.
Specifications for the ASUS UGen300 USB AI Accelerator (UGen300-USB-8G) include:
AI accelerator – Hailo Hailo-10H delivering up to 40 TOPS (INT4) or 20 TOPS (INT8) for inferencing
Memory – 8GB LPDDR4 @ 4266 MT/s
Host interface – USB 3.1 Gen2 (10 Gbps) Type-C port
Power Supply – 5V via USB-C port
Power Consumption – 2.5 Watts (typical)
Dimensions – 105 x 50 x 18 mm
Weight – 145 grams
Fanless device utilizing a substantial internal heatsink to cool the Hailo-10H chip
ASUS asserts that the USB AI Accelerator is compatible with Windows, Linux, and Android operating systems operating on both x86 and ARM host architectures.
It endorses a wide array of both vision and generative AI models, featuring access to over 150 pre-trained models through AI frameworks such as Keras, TensorFlow, TensorFlowLite, PyTorch, and ONNX.
Detailed installation guides for the HailoRT SDK on Ubuntu, along with Python tutorials for LLM, VLM, and speed-to-text functionalities, are accessible via the ASUS documentation.
As highlighted in a prior examination of the Raspberry Pi AI HAT+ 2, while the Hailo-10H may not significantly accelerate LLM workloads, it effectively aids in delegating processes to the USB adapter.
This capability liberates additional memory and CPU/GPU resources on the host, potentially achieving this with reduced power consumption. Consequently, it may serve more as a facilitator for LLMs rather than a direct accelerator.
Nonetheless, it will indeed enhance performance in AI vision tasks and may provide modest improvements with VLMs.
Software Suite
Despite its introduction several months ago, the UGen300 USB AI Accelerator has only recently become available for public purchase.
ASUS presently retails the USB adapter for $299.99 on its online store and on Newegg. The latter also offers the ASUS UGen300 M.2 Hailo-10H module with 8GB priced at $259.99, while a 2GB RAM variant is currently under development but not yet for sale.
Source link: Cnx-software.com.
