> ## Documentation Index > Fetch the complete documentation index at: https://tensorfuse.io/docs/llms.txt > Use this file to discover all available pages before exploring further. # LLM inference on CPUs and Apple silicon > Evaluate CPU and Apple silicon inference with llama.cpp and MLX. Understand unified memory, quantization, memory bandwidth, and local serving limits. An occasional extraction task or local assistant may not need a dedicated cloud GPU. CPUs and Apple silicon can run suitable language models, but fitting the weights is only the first requirement. Evaluate response time, memory pressure, concurrent users, and operating cost with your real workload before treating local execution as production serving capacity. ## How does CPU inference work? A CPU backend uses supported vector or matrix instructions and memory available to the host. Performance depends on the processor, memory bandwidth, thread placement, and implementation of the model's operators. The [llama.cpp project](https://github.com/ggml-org/llama.cpp) documents CPU execution, quantization, and multiple accelerator backends. Quantizing weights can reduce their storage and the bytes read during generation. It can also change model quality and kernel performance. Test task accuracy and latency together; a smaller model file is not enough to establish a better result. On multi-socket servers, record thread affinity and memory placement. Increasing thread count can stop helping when memory traffic or synchronization becomes the limit. Also include preprocessing and other services sharing the host. ## Apple silicon is more than CPU execution Apple silicon combines different processing units with unified memory. MLX uses a shared memory model for CPU and GPU arrays, described in the [MLX documentation](https://ml-explore.github.io/mlx/build/html/index.html). [MLX LM](https://github.com/ml-explore/mlx-lm) builds language-model inference tooling on top of that stack. llama.cpp also provides a Metal backend. A local model running on a Mac may therefore use its GPU rather than only its CPU. State the backend when reporting results. Do not assume the Apple Neural Engine is involved merely because the machine includes one. Unified memory avoids some explicit CPU/GPU transfers and makes a common memory pool available to supported workloads. The operating system and other applications still consume that pool. Leave headroom for runtime state and context growth; swapping can cause severe latency spikes. ## Which workloads make sense to test? | Workload | Useful first check | | ---------------------------------- | ------------------------------------------------------------------- | | Personal assistant | Interactive latency at one active session | | Local extraction or classification | Accuracy and total completion time on representative documents | | Small internal service | Latency with multiple simultaneous requests and other host activity | | Offline batch processing | Work completed during the available window, including failures | Local processing can avoid a remote inference call, but the application may still download models or call external tools. Inspect its actual network behavior when data locality motivates the design. ## When should you move to another backend? Measure cold and warm runs, long-context requests, sustained generation, and concurrency. Record wall power if energy is part of the decision, and use a consistent system boundary when comparing it with cloud or accelerator measurements. If your local setup misses the required response time or capacity, compare a smaller model, a different quantization, or an [alternate hardware path](/docs/guides/inference/hardware/overview). Use the [benchmarking guide](/docs/guides/inference/benchmarking) to preserve the same application target. This page describes CPU and local-device evaluation; it does not add Apple silicon as a Tensorfuse deployment option. ## Related guides Find a model that meets both quality and resource requirements. Evaluate a GPU execution stack when local capacity is insufficient. ## Put this into practice Run the [CPU llama.cpp tutorial](/docs/guides/inference/how-to/cpu-llama-cpp) or [Apple silicon MLX tutorial](/docs/guides/inference/how-to/apple-silicon-mlx) for installation, quantized models, and local API requests.