A low-code project can use AI to draft a page, classify content, propose a database structure, or explain a failed build. Those jobs have different requirements. Choosing one model because it produces an impressive answer to a general question leaves the actual workflow untested. Start with the decisions your project needs to make and the errors your team can realistically catch.

Local and frontier are also different kinds of description. Local describes where inference runs. Frontier is an informal label for models considered highly capable at a particular time; it does not define one hosting arrangement or a permanent quality threshold. The useful comparison is between concrete model and runtime combinations. Our local model planning guide and frontier model workflow guide approach those combinations from complementary directions.

Describe the work before choosing the model

Write a short task specification using real project material that you are allowed to process. For a resource directory, that might be turning an approved description into a title, summary, and three categories. For a website, it might be adapting an existing navigation component while preserving keyboard behavior. Include the expected output format and the evidence a reviewer needs to approve it.

Separate straightforward transformations from decisions involving uncertainty. Shortening a verified description while preserving its meaning has a different review burden from researching whether a technical claim is current. Generating a component has a different failure mode from executing it. A model that performs adequately on one task should not receive automatic approval for the others.

Keep an ordinary manual path in the comparison. If an editor can complete a task quickly with a reusable template, the AI version should justify its extra setup and checking. The outcome to measure is a finished, acceptable artifact, including the time spent repairing it.

Evaluate deployment and capability separately

Deployment determines what you operate

For a local candidate, document the machine, runtime, model files, context settings, and connection behavior. Decide who updates that environment and who helps when it stops working. For a hosted candidate, document the service boundary, permitted input, credential handling, and what happens when the service is unavailable. Avoid treating either arrangement as self-managing.

Running inference locally does not by itself settle every privacy question. Review the surrounding editor, extensions, telemetry settings, backups, and any connected services. Similarly, a remote model should be assessed using the terms and controls of the specific service being considered. Keep sensitive information out of an experiment until that review is complete.

Capability needs evidence from your tasks

Label the outputs by behavior rather than reputation. Did the model retain every required field? Did it invent a product feature? Did the proposed code change unrelated files? These observations are more actionable than calling an answer smart or weak. Save failures alongside successes so the next model comparison starts with useful examples.

Account for context and memory

A model needs enough relevant context to perform the task, but sending an entire project is rarely a substitute for selecting the right files. Include the current component, the relevant design conventions, the requested change, and a small example of the expected result. Remove duplicate logs and stale instructions that might conflict with the actual code.

The official Ollama context-length documentation explains that increasing the configured context length increases the memory needed to run a model. That relationship is a practical reason to test a realistic workload on the intended hardware. A successful short prompt does not establish that a long editing session will fit comfortably.

Try a normal case and a deliberately long case. Record whether the runtime completes, whether the response arrives soon enough for the task, and whether important constraints survive near the end of the conversation. If the long case fails, first narrow the task or organize its context before assuming that a larger context setting is the only solution.

Build a small, repeatable evaluation

Create a fixed packet of representative tasks. For a hypothetical studio catalogue, use an ordinary listing, a listing with missing information, a long description, a conflicting category request, and an input containing instructions that should be treated as source content. These are example test cases, not claims about measured model performance.

  • Specify which fields must exist and which values are allowed before running a candidate.
  • Mark unsupported factual additions as failures even when the prose sounds polished.
  • Record corrections separately from complete rewrites so review effort remains visible.
  • Repeat ambiguous cases to see whether the behavior is stable enough for the workflow.
  • Keep the same task packet when comparing model or runtime changes.

Use a reviewer who understands the intended result. A clean JSON object can still contain the wrong category, and valid HTML can still misrepresent the content. Automated checks are useful for structure; editorial and technical judgment are still needed for meaning. Agree on that division before describing a workflow as ready.

Try the complete editing session

Run a candidate through the work around the prompt as well as the prompt itself. Open the actual editor, load the normal project, make a small revision, inspect the result, and continue with a follow-up request. On a shared workstation, include the other applications people normally keep open. Record interruptions and recovery steps instead of counting only the response that eventually succeeded.

Use the language and content conventions your audience needs. A directory serving several regions may contain names, punctuation, and mixed-language descriptions that were absent from an initial demonstration. Include those materials in the evaluation with a reviewer who can judge them. Do not assume that a result in one language establishes equivalent performance in another.

Also test the handoff to the next person. Can an editor identify the source of a claim and understand why a category was selected? Can a developer see the proposed change separately from unrelated modifications? Prefer an output format that supports that review. If the surrounding workflow hides evidence, improve the handoff before asking a different model to solve the same poorly defined task.

Compare the full operating effort

A useful cost worksheet has more than a model price column. For a local setup, include equipment already available, additional equipment if needed, setup effort, electricity assumptions, maintenance, and the time that other work loses while the machine is occupied. Treat estimates as assumptions and update them after a realistic trial.

For a hosted setup, include the actual service terms you have verified, input and output usage, repeated attempts, integration effort, and review time. Do not multiply a short demonstration by an optimistic volume forecast and present it as a dependable budget. Long inputs, failed attempts, and growing project context can change the workload.

Compare cost per accepted result across a defined batch. Suppose an editor must approve a set of product descriptions. Count the requests, the preparation work, and the minutes required to reach approved copy. That makes a fast draft with heavy correction comparable to a slower draft that needs little intervention.

Use a mixed workflow only when its boundaries are clear

It may be useful to assign routine transformations to one candidate and complex review tasks to another. That is a routing design, not a guarantee that one class of model always handles the easy work. Base each assignment on the evaluation packet and retain a manual fallback when neither candidate is adequate.

Define escalation conditions in terms a person can inspect. Missing required fields, unresolved source conflicts, or an invalid code change are stronger signals than a model's own confident wording. If an automated check fails, preserve the input and error for review. Avoid silently sending a failed local request to a remote service with different data permissions.

The AI API architecture guide covers the integration boundary, while the API integration patterns article explains why request handling needs explicit failure states. Keep the model choice replaceable by passing a defined task contract through that boundary rather than scattering provider-specific assumptions across every component.

Make the decision easy to revisit

End the trial with a brief decision record: the chosen candidate, the tasks approved for it, the tasks still requiring manual work, the evaluation examples, and the main unresolved limitations. Record the exact configuration used. Schedule a new comparison when the work changes or a concrete limitation blocks progress, rather than replacing a stable workflow for novelty alone.

The strongest choice is the one your team can explain and operate. A local setup, a hosted model, or a mixture can be suitable when it produces acceptable work within the project's constraints. Start with one bounded task, measure the complete review process, and expand only where the evidence supports the next step.