How to Integrate AI Features Into an Existing Web Application

Artificial Intelligence is revolutionizing the way people engage with software.

Features that were once the domain of cutting-edge research environments natural-language search, recommendation engines, content generation, document analysis, and conversation assistants are now commonplace in web applications.

For an existing application that fulfills a purpose, the challenge is to determine where AI can truly enhance the experience, and how the technology can be incorporated.

While the technical hurdles for AI-driven software can be considerable, these can often be implemented on a graduated scale through APIs, backend services, databases, or well-designed user interfaces.

The goal in these cases is to determine how best to leverage this technology to achieve a specific objective.

The following is a guide to getting started with incorporating AI into an existing web application, from identifying opportunities to technical and design considerations.

Why Should an Existing Web Application Use AI?

Before embarking on the process of incorporating AI, it is important to set expectations.

AI should enhance an existing capability or fulfill a particular need, rather than being implemented as a novelty.

An e-commerce platform, for instance, may want to utilize AI to suggest products; a customer support platform may want to utilize AI to summarize conversations; and a business management application may want to utilize AI to allow employees to naturally-query data.

The purpose of these implementations should be to improve the experience of fulfilling existing objectives, such as making purchases, providing or receiving support, or analyzing information.

AI can also be used to reduce drudgery in monotonous tasks, by replacing or assisting with aspects such as document review, categorization, summarization, or answering often-asked questions.

That said, not every process is best-suited for AI, and other software solutions or rule-based automation may prove more reliable.

The goal should be to identify the value proposition of a given application of AI.

Evaluating the Current Application

The first major task should be to understand the makeup of an existing application.

Developers must familiarize themselves with the technologies that comprise the frontend, backend, database, APIs, authentication, and other services that comprise the application.

This enables the team to identify opportunities to add AI or a dedicated AI service without overhauling existing systems.

For example, if an application already has a backend API, it may be possible to connect an AI service through that existing API, rather than designing an entirely-new architecture.

The development team should identify or review the following:

• The current tech stack
• The backend API
• The database schema
• Authentication, authorization, and permissions
• APIs for third-party services
• Hosting and other infrastructure
• Performance limitations
• And anything else relevant to the current application’s design and limitations

Security should also be a consideration, especially if the application stores or processes data that should not be shared with unauthorized users.

If an application stores customer information, financial data, business documents, or other restricted information, developers should be mindful where sensitive data is stored and what information is visible to other users.

When it comes to incorporating AI, a feature should be integrated in a way that respects the existing data security and privacy model, rather than using an alternate method of securing or presenting information.

Identifying an AI Use Case

With an understanding of the current application, the next task is to identify a use case that can genuinely enhance the product.

AI has a wide variety of applications, including but not limited to:

• AI chat assistants
• Natural-language search
• Content suggestion or generation
• Text summarization
• Document analysis
• Data summarization or extraction
• Recommendations
• Customer support
• Sentiment analysis
• Image recognition
• Personalization
• And predictive analytics

At this stage, it is typically best to identify one particular use-case scenario that the application can utilize, or to repurpose an existing feature that requires substantial user input or has known limitations.

This can range from an AI assistant for customer support to an AI-enhanced search feature for an e-commerce application.

It is important to build an MVP (Minimum Viable Product) for a given application of AI, test it, and refine it before potentially introducing additional AI-driven features.

Selecting an Appropriate Technology

Different applications of AI may require different technologies.

A chat assistant, for example, may have different needs than a recommendation engine.

Some AI applications may require custom-trained models, while others can utilize third-party APIs for content suggestion, image recognition, document analysis, and similar tasks.

AI technology falls somewhere between a rule-based program and a natural language-modeling application.

The business should identify what it wants the technology to do, rather than focusing on the technical details of how it may be accomplished.

At this stage, different options can be compared according to factors such as:

• Accuracy
• Cost-effectiveness
• Speed
• Privacy and security implications
• Scalability
• Compatibility with existing technologies
• And control or customization

The ideal choice will typically be the simplest technology that can fulfill the requirements of a given application.

Connecting AI Through the Backend

For an existing application that utilizes a backend API to fulfill requests from the frontend, the AI service should ideally be connected through the backend as well.

A basic architecture for such an application would be:

User > Frontend > App. Backend > AI > Backend > Frontend > User

This design has a variety of advantages, not least of which is that the application’s backend can handle or restrict access to the AI service.

The application’s backend can also verify the request, analyze any required information, send a request to the AI service, receive a response, and present the response to the user in an appropriate format.

Crucially, this approach keeps API access and other security measures within the application, rather than exposing them to the frontend or users.

Another advantage is flexibility.

If a business wishes to change AI services or migration between different models, this can typically be accomplished in the backend, rather than rewriting instructions in the frontend.

The application’s backend can also include dedicated features for the AI service, such as a script that accepts a query, sends it to the AI service, verifies the response, and logs or tracks the request for user-facing display or internal monitoring.

Connecting AI With Business Information

AI is a tool that can be leveraged to analyze, retrieve, and suggest information that an application or its users need.

That functionality is tremendously valuable if the AI service can be connected to the application’s existing data.

A chat assistant, for instance, may be able to utilize information from product documentation, databases, or other resources to provide more accurate or relevant information about the application or connected business.

This has the potential to dramatically improve the capabilities of AI-driven applications, including chat assistants and document or information retrieval tools.

One approach to this is RAG, or Retrieval Augmented Generation.

This technique utilizes databases to analyze and retrieve relevant information that can be used to inform the AI model.

For instance, if an application has access to thousands of documents, a user may be able to ask a question about those documents.

Rather than having the model attempt to learn and apply that information, the RAG application will utilize a search function to find relevant information from the documents, then apply filtering or formatting before sending that information to the model, which will then generate a response.

This is a valuable technique for a variety of applications that involve knowledge-bases, documentation, customer support, and similar functions.

It can be applied to company knowledge-bases, product documentation, and information retrieval or customer support applications, among other potential uses.

That said, it is important to remember that appropriate security and permission controls must be applied to any AI-driven query feature.

A user should only be able to request or receive information that they are already authorized to view, rather than granting blanket access to a database based on their ability to ask a question.

A similar principle should be applied to AI-driven applications that utilize document analysis.

Protecting Users’ Data and Permissions

The introduction of AI services opens up additional opportunities for data breaches and unauthorized access.

The permissions model and data security of an application should be applied consistently across its features, including AI-driven tools.

For instance, if an application has an administrative dashboard that can be accessed by managers but not standard users, an AI assistant within the application should have the same limitations.

The AI assistant should not have unrestricted access to the application’s database and should not attempt to determine what information a given user can and cannot view, that responsibility should lie with the application itself.

Securing data and reducing the attack surface are paramount, and AI services should be utilized responsibly within an application’s security profile.

Designing Better Prompts

For AI-driven applications, prompts are critical instructions that determine what the model will do.

A production-ready prompt typically contains additional instructions beyond basic questions or guidelines.

It may also include system instructions, information about the user’s request, relevant information retrieved from a database, formatting or stylistic instructions, behavioral instructions, or other elements.

An AI-driven support application, for instance, may have the following instructions:

• Follow instructions from the user
• Provide information that has been approved for public access
• Avoid sharing information about products that the business does not produce
• Provide concise responses
• If you are unsure of a response, ask for clarification
• Avoid sharing internal or sensitive information
• And escalate certain requests to human moderators

These instructions can help to standardize and improve the responses that a user receives, but they should not be considered an assurance of safety or accuracy.

A malicious user may be capable of bypassing certain restrictions by attempting to manipulate the model.

Important security and permission features should always be applied at the application layer, rather than relying on prompts to prevent malicious activity.

The contents of a model’s response should never be used to execute commands, display untrusted HTML in a browser, write information to a database, or perform potentially-dangerous operations.

AI Response Content Should Always Be Considered Untrusted Content

Some AI applications go beyond simply responding to user input.

They can be programmed to execute commands and perform additional feats.

These are sometimes called AI agents.

Such applications can perform a variety of tasks, including analyzing data, searching databases, and even executing commands.

The same precautions that are taken with standard prompts still apply, but additional considerations should be made when designing such applications.

A user-facing AI assistant, for instance, should not be able to gain unrestricted access to a database or application without explicit permission in a manner that goes beyond standard application permissions.

Additional considerations should be made when designing such applications to ensure that proper safeguards such as permission controls, input validation, and user confirmation are in place for high-risk operations.

Enhancing the User Experience

An existing web application should be intuitive and easy to use.

Introducing a new AI tool or feature should not drastically alter the manner in which an application is used or understood.

That said, the feature should be intuitive and accessible to end-users.

An online learning application, for instance, may utilize an AI assistant to help answer questions or provide relevant information.

A user does not need to visit a separate page or learn an entirely-new application they simply ask the AI assistant for information and receive it within the same interface in a helpful format.

A similar principle applies to customer service applications and other features.

The end-user experience should always be considered when designing or integrating an AI tool, with particular consideration given to how much control or input the user should have.

In some cases, this may entail giving users the ability to modify, cancel, or resubmit prompts or requests.

Human Moderation Can Be Valuable

AI assistants can be extremely valuable, but they are not always accurate or appropriate for a given task.

It can be valuable to incorporate human moderation for especially-important or sensitive tasks, or to reduce the risk of errors.

AI assistants can summarize contracts, highlight potentially-illegal transactions, recommend or draft customer responses, escalate sensitive requests, or perform a variety of other tasks.

The application can inform an appropriate human moderator when such circumstances arise, rather than relying on the capabilities of an AI model alone.

This also reduces the risk that the model will provide inappropriate or harmful content or attempt to obfuscate the matter.

This approach can also serve as a valuable feedback loop.

If users are routinely correcting an AI response, programmers can investigate why that is happening and determine if it is due to a problem with the AI model, prompts, information retrieval methods, or other factors.

Testing AI Applications Differently

Testing software typically entails reproducing a specific set of actions and ensuring that the same result or behavior occurs each time.

AI-driven applications, however, require additional considerations.

Responses can vary considerably, particularly for seemingly-similar requests.

A response may appear to be accurate but could still have serious flaws or fail to address a request.

Testing such an application entails additional considerations, including but not limited to, evaluating the accuracy of a given response, its relevance to the request, consistency, formatting, potential failures or limitations, and more.

Malicious requests should also be tested to ensure that prompts cannot be used to circumvent an application’s permission model or cause the AI to expose sensitive information.

Additional testing should be conducted to ensure that responses do not attempt to execute potentially-dangerous commands.

AI security testing should be an ongoing process, as an application’s data, prompts, users, or other factors may change after launch.

Monitoring AI Performance and Costs

Once an AI-driven application or feature has launched, it is important to monitor it for performance and potential issues.

Traditional applications typically track performance based on server response time, database activity, errors, uptime, or other similar metrics.

AI-driven applications should track additional metrics that are relevant to the specific application, such as the number or rate of requests, the response time of the AI itself, errors, token usage, expenses, caching behaviors, and more.

Expenses can also be a significant concern, particularly with large-scale applications that receive thousands or millions of requests per day.

Some AI technologies may charge based on the number of tokens processed in a request, which can add up if a feature receives substantial traffic.

Caching, optimizations to prompts, shorter model responses, reduced request volume, and similar approaches can reduce costs.

Monitoring can also help ensure that users are engaging with the feature as intended.

It can be valuable to determine if an AI feature has met business objectives, including encouraging engagement or being utilized. If an application’s AI assistant receives only a handful of requests per day after launch, it may not be fulfilling a need or difficult to discover, rather than being a fundamentally-flawed concept.

A Proof of Concept Can Help Reduce Risks

Many organizations hesitate to adopt new technologies due to the risks and costs of implementation.

New applications of AI, in particular, can be extremely expensive and time-consuming to develop, only to fail to meet end-user expectations.

Before launching a feature or a significant investment in AI, it can be valuable to construct a proof of concept that demonstrates that the application fulfills requirements and has practical value.

For an AI assistant, this could entail constructing an MVP that utilizes limited information and testing its accuracy, response time, potential issues, and overall value.

This can reduce the risks, costs, and time associated with adopting AI while still realizing its benefits.

The limitations of a given architecture can also be identified and addressed, including if the application requires significant changes to support a new AI service.

Additional Common Mistakes

Several pitfalls should be avoided when adopting AI within an existing web application, including but not limited to:

• Implementing AI without a specific need or objective
• Giving an AI service unnecessary or excessive access to data or functionalities
• Failing to consider the quality of data
• Lacking safeguards for AI-generated responses
• And underestimating costs

AI should address a specific need or task, rather than being implemented for its own sake.

An AI assistant that solely exists to demonstrate that the application utilizes AI, for instance, is unlikely to have long-term value.

The AI service should only receive the permissions and data that it needs to fulfill its purpose.

The data quality is also critically important a model will provide poor results if an application’s database contains irrelevant or low-quality information.

Finally, applications must consider the practical implications of AI-generated information.

If an AI assistant provides information that should not be shared with the user, it compromises both privacy and the user experience.

A similar consideration applies to responses or actions that should require additional review or safeguards, such as financial transactions, before being enacted or displayed.

Conclusion

Integrating AI into an existing web application does not require overhauling the application’s core technologies.

In most cases, AI can be implemented through a combination of frontend, backend, and database modifications, or through dedicated AI tools or APIs that connect to existing databases and APIs.

The most important step is to identify an objective or need that AI can truly fulfill, after which developers can choose an appropriate technology, design the architecture, incorporate data, implement permissions and security, design a user-friendly interface, test the product, and release it to users.

AI features should be viewed as ongoing developments that require updates to prompts, information, applications, and more, rather than a one-time implementation.

The most valuable applications of AI are those that fulfill real-world needs, rather than being gimmicks that provide little tangible value to a business or its users over time.

As AI evolves and improves, existing web applications have the opportunity to transform themselves into powerful new experiences without abandoning core technologies, data, and user expectations. A thoughtful combination of AI and application development can turn an ordinary web application into a truly remarkable one.