Open sourced code
I released quite a few projects as open source – the following ones are the most notable:
- poly – an LLM server using llm and ggml. (Rust)
- rv32jit – a just-in-time compiler for RISC-V (rv32imc) aimed for usage on ESP-32. (Rust)
- warp – a Mac and iOS app for effortless analysis and transformation of large data sets. (Swift)
- cactus – a software 3D renderer, ported from idx3dIII. (C++)
- tjshow – software for show control. (C++)
Contributions to existing open source projects
I contribute to various open source projects (usually on GitHub). Interests include AI/LLM inference, WebGPU and previously mobile applications. My contributions are primarily written in Rust. A full and current list can be found here.
2023
- Implemented Metal GPU inference for rustformers/llm, a crate wrapping ggml for LLM inference. This leads to a 31% speed up (and about a 13% power usage reduction) running 13B LLaMA models on an M1 Max.
2022
- Maintainership of webonnx/wonnx, a library for cross-platform neural network inference on the GPU. Contributions include:
- Implementing shape inference
- Adding all operators needed in order to run BERT models
- Various optimizations including intermediate buffer coloring, constant folding
- Implement an IR (intermediate representation)
- Add a command-line interface