Contact us

    Talk to our Experts

    +1 920 303 0470 1302 S Main ST, Oshkosh, WI 54902-6518

    Getting Ready for AI in Packaged Applications and What can I do now?

    What is AI?

    Terms like Artificial Intelligence (AI) and machine learning are often used in a variety of contexts with little clarity. In addition, their scope is often exaggerated as being fundamentally different from where we are today. Reality is quite different though where AI based algorithms simply represent the next generation of our solutions. The basic differences are:

    • AI Algorithms rely heavily on statistics and probabilities
    • Machine Learning concepts are incorporated to fine-tune the statistical models. In principle this is similar to the traditional DevOps cycle except that some parts of that flow may be done by machines.

    Thus, these terms do not imply a fundamental shift. But relying on statistics open up unique opportunities and challenges.

    Challenges for Packaged Applications

    We have all seen papers and presentations about the potential and reach of AI-based solutions for business problems. Packaged systems like SAP, Oracle, Blue Yonder often a unique challenge in this space. If we use these systems then while we may become users of the AI solutions, we will not have access to the raw source data or the core capabilities of the underlying algorithm. For example, while we may use “Google Photos”, we do not have access to the raw underlying models.

    This offers some unique challenges:

    • For several use cases, we have to wait for the system to provide AI solutions.
    • We need to explore and understand how the AI solutions will fit into our business environments before the solutions become available.

    The latter is an especially important consideration for several businesses where end-to-end system testing is critical to the system acceptance. Traditional test cases expect an exact output for a given input. As the AI-based algorithms rely heavily on statistics and existing data; the exact output is difficult to predict in a given environment.

    It is important to have these discussions in your organizations before these algorithms are released. This is to make sure that the internal QA and audit teams understand these nuances regarding AI solutions and the expected testing protocols. For example, in some cases the success criteria may need to be probabilistic rather than exact.

    What can I do now?

    Even though terms like Chat-GPT and AI have become household terms recently, the industry had been making advances in this space consistently. AI has been improving over the last several decades and thus several domains have become quite mature. We have been consuming AI services for some time now.

    Several modern systems including systems in the Blue Yonder ecosystem offer modern hooks where we can interact with the system using API layer. This capability empowers us to use certain AI-based solutions now.

    Computer Vision

    Using Computer Vision to recognize images and patterns is a mature domain. We have been using it for several years as end-users. It is interesting because it is ubiquitous and quite accurate. Furthermore, the learning protocols required to train and fine-tune these algorithms are quite simple.

    On the other side, it is simple to integrate into an ecosystem. All it requires is an imaging device like a camera (video or still). If the information represented is simple, we can then easily integrate that with any system. Following are some use cases that we can easily integrate into Warehouse Management Systems:

    • Cycle Counting

    Cycle counting is an expensive use case for WMS systems as it takes away resources from other critical use cases. These are often mandated so there is little flexibility in this area. Over time WMS systems have creatively introduced features like “Count Near Zero” to take advantage of certain organic conditions which could serve as counts for the purpose of satisfying the overall business requirements.

    While counting may be a labor-intensive use case, a subset of these use cases could be quite simple, for example:

    • Seeing which locations are empty.
    • Detecting Pallet and full case locations to count the inventory.
    • Counting pieces in locations where inventory may be stored in a well-organized fashion.

    Thus, there is an opportunity to address a subset of the overall counts using a creative solution based on computer vision. We can imagine, for example, cameras attached to fork-lifts or other devices in the warehouse which could utilize this technique to satisfy the cycle count requirements.

    At Smart IS, we developed a model for this purpose and trained it for this use case. We then tested it on certain real-world scenarios and as you can see our recognition was appropriate for use in real-world conditions:

    • The first image properly detected that there is a single location that has two empty spots and two spots contain a box.
    • Second image properly analyzed the shelf to detect that it was empty.
    • The last image demonstrates that we can look at a shelf from a distance and it can be trained to understand the location boundaries and inventory characteristics.

    A critical component of Smart IS solution is to incorporate machine learning that allows us to improve the accuracy of the solution and address any unique and unforeseen situations.

    • QA Audits

    QA Audits offer an interesting challenge as well. For some industries, like life sciences, they are critical to the overall solutions. When performing certain VAS (Value Added Services), we often require that the space where we are performing that action must be clean before a user starts their work. QA may require manual checks and double-checks for this — which increases the overall cost and negatively impacts the throughput.

    We were able to train our model to detect this condition quite accurately. Moreover we can improve the solution further by storing the image alongside of the VAS operational record.

    Other types of QA and safety conditions can be handled easily as well, for example:

    • Are people wearing the required safety gear?

    • Is a space getting congested?

    Yard Management

    Yard Management use cases can take advantage of the advances in this space as well. For example:

    • Capture the license plate to drive the trailer check in process.

    • Find the empty parking spots.
    • Utilizing Video Feeds

    We are not restricted to still images in this space. We can take advantage of video feeds as well. The following feed is from our Pakistan office where we trained our model to see who was playing ping pong and who was standing on the side:
    • Incorporating Machine Learning

    In order to use computer vision to support business use cases, we need to incorporate machine learning as we need to constantly incorporate the feedback.

    At Smart IS we follow this approach to fine-tune our models where we incorporate the feedback to improve the quality of our models.

    Using Large Language Models (LLM)

    All packaged applications provide a framework that allows for creating simple pages and reports to provide data to the end users. For example, in Blue Yonder WMS we used to have Data Driven Applications (DDAs) and the later versions support “Page Builders”. The basic problem that such front ends solve is providing critical data to the key users easily.

    As LLMs are becoming mainstream, at Smart IS, we have developed a framework that empowers the end users to interact directly with the back-end application without needing an intermediate report or screen. The users can simply ask the questions in natural language.

    Some distinguishing features of our solution are:

    • It can run as a stand-alone chatbot.
    • It can run as a teams chatbot.
    • It runs “MOCA Commands” to provide data from the actual WMS system.
    • It provides a framework to act on the data as well.
    • The chatbot runs in the context of the user and thus respects the security regime.
    • It can return the results in a variety of formats, for example raw data or as a chart.
    • It can tell the user how it answered the question.
    • It supports Machine Learning to improve the model.

    Below, I am illustrating some interactions with our chatbot running in teams:

    • The connection with teams is in the context of a BY WMS user. As the interaction starts, it will prompt for the credentials:

    When user starts the team conversation, if the user is not logged in, it will prompt for the credentials

    • The users can then interact with the chatbot in natural language. The natural language prompts map to MOCA Commands to provide the response. We support several popular LLMs to support the natural language prompts.
    • The response can include multi-media as well.

    Natural Language prompt runs a MOCA command and can return data as a grid as well as a chart

    • The ecosystem can tell us what did it do under the hood to return the data.

    We can ask the model to tell us how it came up with the answer

    • The ecosystem supports a context concept. After asking a question where it shows a grid — I can ask about the response. For example, after asking about the work, I can ask:

    The response explains one of the rows in a grid

    Smart IS provides a complete system to support this solution. Our approach in this space is as described below:

    Smart IS ecosystem to support queries against BY WMS using LLM

    This shows that the natural language query is parsed in the cloud using any popular LLM model. That is then passed to the BY WMS system that is running behind a firewall through Smart IS OOGY solution.

    You can see our chatbot features in detail here.

    Document Management

    Complex system implementations including ERP, WMS, and WES systems require extensive documentation during the implementation phases. While the documentation is quite extensive and elaborate it is made up of hundreds of documents spread over several locations. The end result is that the documentation does not serve is required purpose later during upgrades or other initiatives.

    We have seen several solutions emerge in this space over the last few years that could fill this gap for example:

    • Summarize sections of the document. For example, asking about how to process a specific order type from an elaborate solution summary.
    • Converting comprehensive documents into easily consumable FAQs.
    • Summarizing documents like release notes from the vendors.


    Packaged applications offer unique opportunities and challenges in the overall domain of AI initiatives. Such applications often have closed ecosystems that govern the raw data but still offer open APIs for external systems to connect to them.

    As AI-based core algorithms are released over time, the users of such systems have some opportunities and critical responsibilities in this space to be ready for what is coming.

    This blog went over some of these concepts and also some use cases that could be employed today.

    This was the summary of Saad Ahmad Executive Vice President Smart IS presentation at the Blue Yonder Icon Conference this year held from May 13 2024 to May 16 2024. The session’s title was “Image Recognition: Supporting Blue Yonder Use Cases”.

    Originally Published At:

    Leave a Reply

    Your email address will not be published. Required fields are marked *

    Recent Stories

    Copyright © 2024 Smart-IS International. All Rights Reserved.

    Microsoft is a registered trademark of Microsoft Corporation Oracle, JD Edwards and Peoplesoft are registered trademarks of Oracle Corporation. Blue Yonder is a registered trademark of Blue Yonder Group Incorporated. All trademarks are property of their respective owners.