Six Challenges To Tackle When Embarking On Your AI Journey

This blog post was co-authored with David Mosen.

A lot has been written about how AI is the next big thing, how it is revolutionizing businesses across industries, even comparing its transformative power to that of electricity 100 years ago. These are catchy headlines that seem in stark contrast with others that predict and observe staggering failure rates for AI projects. So, what is the source of such an apparent contradiction? In this article, we will explore some of the difficulties derived from the particularities of AI projects that can eventually keep them from being successful, from a technology and project management perspective.

Just as with software projects, AI project management requires proper execution of processes that address business concerns, like requirement, risk and release management. Likewise, causes for failure are mostly shared, including poorly defined goals, miscommunication, insufficient end-user involvement and improper assessment of available resources. On top of that, AI projects face additional hurdles, arguably arising from the introduction of data science into the mix, with its data requirements and the novelty of the methods often involved.

We have compiled a list of the six main challenges that we have been observing during the delivery of 100+ AI projects:

Challenge 1: Trying to run before you can walk

Digitalization precedes data-based projects, like those involving AI. Going from analog to AI is sure to be a treacherous journey, assuming that it is a possible and sensible one. Granted that we are at a time when most large businesses have a clear idea of whether, how and to what extent they can benefit from digital infrastructure. However, it is also true that most are still not data-driven, a last leap that often requires a change in leadership and a great deal of flexibility.

Challenge 2: Stretching the short supply

Data science and AI expertise is not only scarce, but also dispersed in the knowledge space, in the sense that it is still a broadly defined domain that gathers from several better-delimited fields like statistics and software engineering. From a project personnel perspective, this means that single profiles which perfectly match the required expertise might not always be available. Consequently, team harmonization and coordination take a more prominent relevance.

Challenge 3: Communication gap

Together with business and software engineering, data science represents the third leg of an AI project; an addition that introduces further complexity to the interactions across knowledge domains, on top of its intrinsic intricacies. This calls for even further efforts on communication and conflict management, as well as change management.

Challenge 4: You can't assess your tech cake and eat it too

AI projects require thorough, yet limited-in-time, technology assessment. The AI and data science landscape is still developing at an accelerated pace, with not just new tools and services, but also new paradigms. Thus, projects will face compromises: consider the best balance between cutting-edge and maturity; and deciding whether to explore less options in more depth, as opposed to a wider range of options at a higher risk.

Challenge 5: Locked-in

Another consequence of a yet-maturing AI environment is the high probability of any given technology or vendor to quickly grow out of favor. This situation is combined with the fact that large cloud providers are rapidly becoming the go-to vendors for full-cycle data science development. Overall, this makes it important to carefully assess the possibility of lock-in and the level of portability when choosing each of the pieces of the tech stack.

Challenge 6: Data is right, aligned

Software developed within the scope of a project is required to be aligned with business objectives. So does data that is gathered and prepared. Yet, while software requires engineering processes, data requires processes that mix engineering and science, including collection, exploration and transformation. Checking for data availability, completeness, suitability and unbiasedness lean more towards a science approach.

Seek professional help

Whether you decide to undertake the journey in-house or to seek the expertise of an AI consulting services partner, the above challenges have to be addressed to arrive to a successful AI project. A success formula will have to bake in a proper AI mindset that includes the environment, processes, people, data and technology into its project execution and management.

Originally appeared in the We Are Developers Magazine.