Built-in AI or a standalone platform? How to tell what you need

How do you know when the AI in your existing software is enough?

Built-in AI is a good fit when it can handle the task with the information and controls it needs. A standalone AI platform becomes worth exploring when a defined requirement remains unmet, such as a specialised workflow or more control over the AI model.

Start by making that requirement specific. What must AI produce or change, and what would make the result acceptable? Those answers give you something concrete to test against.

What do built-in and standalone AI mean?

Built-in AI refers here to capabilities delivered within an application or software ecosystem your teams use. A standalone AI platform is a separate environment you adopt for AI work, which may connect to those existing applications.

Choosing a standalone AI platform does not mean your team has to manage the servers running the models. With a managed service such as Amazon Bedrock, AWS manages that infrastructure. Your team or an implementation partner still builds and maintains the application and its connections to your business systems.

Start with a task you can test

Consider a hypothetical release briefing. An engineering manager needs a draft summarising Jira work and release documentation, with links to the supporting records.

Test the available AI against that task using real examples. Include missing or contradictory information so you can see whether it flags uncertainty or produces a confident answer anyway. Define what the reviewer must check before using the briefing.

A later requirement might involve checking an internal database and updating another system after approval. That extension needs a separate test of integration and action handling. A convincing summary alone gives you little evidence about either.

Check what the built-in AI can connect to

Needing information from several systems is a reason to examine integrations. It does not automatically establish a need for a standalone AI platform. Check whether the built-in AI available within your software supports the sources involved.

Atlassian Rovo illustrates this. Its Teamwork Graph connector types include synced connectors that store and index source content, and direct connectors that retrieve content live through the source provider's APIs. The connection method affects how information becomes available to AI.

For the release briefing, check whether the relevant records are accessible through a supported connection, how changes reach the AI and whether each user sees the appropriate information. If an essential source remains inaccessible, record that as a specific gap to investigate in other platforms.

Name the control you are missing

A separate platform may be worth exploring when the built-in options cannot support the way the task must run.

Microsoft's comparison of agent approaches makes this distinction concrete. Declarative agents use Copilot's models and orchestration, the logic coordinating the agent's work. Custom engine agents allow developers to select their own models and orchestration. Both approaches can appear within Microsoft 365.

Describe the limitation before proposing an alternative. For example, specify the release checks the built-in tools you are evaluating cannot perform, or the model evaluation requirement they cannot support.

Compare the work after launch

Include ongoing support in the comparison. Who maintains connections when a source changes? Who tests the briefing after an AI update, investigates incorrect output and supports users? Check how those tasks fit your current team or the service you would buy.

Microsoft's cost planning guidance includes usage, hosting, monitoring and support alongside licences. Use those categories to compare the complete application, including any services it depends on. An existing software subscription alone does not tell you the full cost.

AWS's Bedrock security documentation describes a shared responsibility model. AWS protects the service infrastructure, while the customer's responsibilities depend on the service and the data involved. Ask the provider to make that division explicit for the application you plan to run.

Factor in AI token costs per completed task

Tokens measure the input an AI model processes and the output it generates. When usage is charged per token, extra context, additional steps and retries can increase the cost.

Test both options on the same task and required output quality. Compare token consumption and pricing, then calculate the cost at your expected monthly volume. Check what the subscription includes and how usage beyond that allowance is billed.

A practical guide to the choice

If a test briefing is incomplete because its source documentation is outdated, fix that information before retesting. If a required action is unavailable, investigate whether configuration, a supported extension or a separate platform can provide it.

How Idalko helps with AI in Atlassian

At Idalko, we assess your Atlassian data, information structures and access controls, identifying what needs cleaning up or reorganising before introducing AI. We also configure Rovo around relevant use cases, using Search, Chat, Agents, connectors and AI-enabled workflows where appropriate.

Explore how Idalko prepares and configures Atlassian environments for Rovo.