Ideas on low bit architectures

• 29 July 2025
• tech,
• fpga,
• dnn

Research ideas #

References #

• BitNet: Scaling 1-bit Transformers for Large Language Models
• BitNet v2: Native 4-bit Activations with Hadamard Transformation for 1-bit LLMs
• BitNet b1.58 2B4T Technical Report
• The Era of 1-bit LLMs: All Large Language Models are in 1.58 Bits
• ParetoQ: Scaling Laws in Extremely Low-bit LLM Quantization
• TernaryLLM: Ternarized Large Language Model

• Previous: Hardcoded Inference on FPGA