J
Jinfer
AI inference engine for the JVM—chat, vision, audio, embeddings, and TTS in a jar
Visit Jinfer ↗
link: rel="ugc noopener"
Jinfer is an inference engine purpose-built for the JVM, enabling AI capabilities without Python runtimes, ONNX, Docker, or sidecar processes. It supports chat, vision, audio, embeddings, reranking, and text-to-speech functionality with pure-Java tokenizers and quantized matrix multiplication kernels.
The stack is natively designed for the JVM rather than adapted to it, with support for llama.cpp and HuggingFace model formats. Everything runs as a self-contained JAR file.
Discussion