Artificial Intelligence (AI) is becoming as much of a force for world progress as software. However, unlike traditional software engineering, AI engineering introduces a variety of new challenges. Specifically, developing AI systems involves a large amount of design choices with uncertain implications. How can we choose the best data and modeling methods to achieve our goals? How can we evaluate our systems and identify issues with them? How can we form a global understanding of not just our own systems, but the field of all possible AI research as a whole?

Inspired Cognition (IC) is a new company formed to provide tools that work together with developers to guide them through the thorniest challenges in developing AI systems. For instance, the current software that we support includes:

• DataLab: DataLab is a development library that allows you to efficiently locate, assess, and manipulate datasets to improve your systems. DataLab supports 800+ datasets for 20+ tasks, and it is easy to add your own by contributing open-source. DataLab was selected as an outstanding system demonstration at the 2022 version of ACL.

• EaaS - Evaluation as a Service: EaaS provides “evaluation as a service,” allowing you to assess the quality of machine-generated texts using a number of state-of-the-art techniques through a simple API library. It supports a wide variety of evaluation metrics, from simple mainstays to state-of-the-art learned metrics, all available through an easy-to-use and efficient API.

• ExplainaBoard: ExplainaBoard provides “explainable leaderboards” that allow you to compare and contrast the performance of your systems on a fine-grained level, identifying pain points and improving performance effectively. You can easily get a taste of how it works through the web interface, power users can use or contribute to the offline SDK. ExplainaBoard was selected as the best system demonstration at the 2021 version of ACL, the flagship conference of natural language processing.

We will work together with the AI research community, individual developers, and companies to help realize this vision of building AI development tools for all. If you’re interested in being involved, please take a look at our tools above for ways to contribute, or get in contact with us if you’re an organization interested in working with Inspired Cognition to speed the development of your AI systems.

This is just the beginning of our journey, and we’re looking forward to even more inspiration to the road ahead!