01 · Architecture
What does “local AI” actually change?
A local model performs inference on a computer you control instead of sending the prompt and source material to a hosted model API. That can reduce third-party exposure, give you more control over the model and runtime, and keep working when an external service is unavailable.
It does not automatically create privacy, security, accuracy, or compliance. Those are properties of the complete workflow: device access, storage, logs, backups, network behavior, retention, people, and professional review.
Local can reduce
- Model-API transmission
- Dependence on one hosted vendor
- Recurring per-token charges
Local does not remove
- Device and account risk
- Hallucinations and omissions
- Your duty to review the work
02 · Boundaries
Follow the complete data path—not the marketing label.
“The model runs locally” only describes one component. A desktop app may still call home for updates, analytics, crash reports, license checks, cloud sync, web search, transcription, plugins, or remote model fallback.
- Identify the exact app and runtime.
- Identify where the model artifact came from and where it is stored.
- Inspect network behavior, logs, temporary files, sync, and backups.
- Confirm who can access the machine and retained outputs.
- Test with synthetic or public material before sensitive data.
What would “offline” require?
More than switching off Wi-Fi once. The workflow needs an installed model artifact, a runtime that does not require a remote service, disabled remote fallbacks and integrations, and a deliberate policy for updates, backups, exports, and later reconnection. Verify the actual system; do not infer it from a product label.
03 · Hardware
Why does unified memory matter so much?
Apple silicon lets the CPU and GPU use one memory pool. That makes Macs unusually approachable for local inference because model weights do not need to be copied into a separate, smaller graphics-memory pool. But the model does not get every advertised gigabyte.
The operating system, open applications, model runtime, working context, and generated tokens all consume memory. The checker therefore rounds down and treats fit as a planning estimate—not a benchmark.
Memory configurations covered by the checker
8, 16, 18, 24, 32, 36, 48, 64, 96, 128, and 192 GB, plus any custom amount. These span Apple-silicon configurations across multiple Mac generations and product families; they are not a claim that every amount is sold on every current Mac.
Use the full memory checker →How should I interpret the model ranges?
They are cautious Q4 artifact-fit ranges. Roughly: 8 GB starts around 3–4B; 16–18 GB around 7–9B; 24 GB around 14B; 32 GB around 20B; 36 GB around 27B; 48 GB around 32B; 64 GB around 40B; and 96 GB or more can plausibly hold a 70B-class Q4 artifact. Exact artifacts, context, runtimes, and other apps can move the line.
Does more memory make the answer better?
No. More memory lets you load a larger artifact or use more context. It does not establish that the larger model is better for your task, that it follows your format, or that it is fast enough to be useful.
04 · Models
Model size, quantization, and context are separate knobs.
Parameters
A “7B” or “70B” label describes an approximate parameter count, not a quality score. Architecture, training data, post-training, tool use, and the task itself matter.
Quantization
Quantization stores weights at lower precision to reduce memory and often improve practical speed. “Q4” is a family of roughly four-bit approaches, not one universal file format or quality level. Always name the exact artifact.
Context
The prompt, source documents, conversation history, and generated output occupy working memory. A model may load successfully and then run out of headroom on a long document.
Runtime
The runtime is the software that loads and executes the model. MLX-based tools, llama.cpp-family tools, and packaged desktop apps can use different formats, kernels, defaults, and memory behavior. A model name alone is not a reproducible setup.
Fit ≠ speed ≠ quality ≠ suitability
Those are four different questions. The memory checker only estimates the first. A retained benchmark can measure speed. A scored field test can evaluate quality. A qualified professional and the organization’s controls determine suitability.
05 · Workflow design
Start with the customer’s real task.
The strongest first candidates are repetitive documentation tasks with known source material, a standard output, and a qualified reviewer. The model drafts or transforms; a person owns the conclusion and final work.
Meeting notes → a standard record
Test names, dates, figures, commitments, action owners, omissions, and unsupported additions.
Approved notes → a first draft
Require source traceability. Check every factual statement and remove content the source did not support.
Intake → a known internal format
Score missing fields, negations, identity errors, dates, and sensitive-field handling.
Document triage before full review
Use it to prioritize attention, never to replace the complete professional review. Score omissions and cross-reference errors.
Poor first candidates include autonomous decisions, final legal or clinical conclusions, work without a reliable source, and any task where no qualified person can define correctness.
06 · Evidence
What turns a claim into evidence?
A result becomes inspectable only when the exact machine, operating system, runtime, settings, model artifact, safe fixture, prompt, raw outputs, timing method, errors, and reviewer rubric are retained.
Calculated
Assumptions are stated. Useful for planning, not proof.
Predeclared
The task and scoring plan exist, but the run is not complete.
Retained
The run artifacts, failures, timing, and evaluation can be inspected.
Ground Floor currently publishes three practitioner test protocols. They are plans, not completed practitioner studies. Attractive demo outputs are not silently promoted into results.
07 · Testbed
What is the larger hardware lab trying to learn?
The declared Ground Floor testbed includes two 128 GB M5 Max MacBook Pros connected over Thunderbolt 5. The reason for a two-node system is not that ordinary users need one. It creates room to study larger artifacts, distributed inference, interconnect overhead, and whether additional hardware produces useful workflow gains.
Evidence boundary: hardware inventory is a declared configuration. Latency, bandwidth, throughput, scaling, energy, and quality claims only become measured when the exact command, settings, repetitions, logs, and raw results are retained with the run.
Why not publish a giant model leaderboard?
Leaderboards age quickly and can hide the variables that matter: exact artifact, prompt, context, runtime, task, rubric, and hardware. Ground Floor prioritizes reproducible task-level evidence over a broad ranking assembled from mismatched sources.
08 · First run
A responsible first experiment takes about one narrow question.
- Choose one repetitive task and define the required output.
- Check your Mac’s unified memory and choose a conservative model class.
- Install a known runtime and record the exact model artifact.
- Use synthetic or public source material.
- Save the prompt and every raw output—including failures.
- Score facts, omissions, unsupported additions, format, edits, and time.
- Decide whether the result merits a larger test. Do not jump directly to sensitive production work.
The editorial rule
Task first. Demonstrate what it does. Tell the whole material story—including the limits.
That is why this page keeps architecture, hardware, model fit, workflow value, evidence, and risk together. None of those pieces is the whole answer by itself.