Sherpa UPDATE - 11 AUGUST 2023

In This Page:

Sherpa UPDATE - 11 AUGUST 2023#

Amir Feizpour

Summary#

Amir Feizpour presented an open source project that aims to revolutionize knowledge jobs by leveraging artificial intelligence (AI). The project focuses on operationalizing knowledge, enabling machines to read and provide real-time information. The goal is to enhance communication, collaboration, and project management by utilizing machine learning models with language and reasoning skills.

Topics:#

Project Overview

  • Machine learning agents collaborating to find and provide necessary information for knowledge jobs

  • Integration of AI into workflows to streamline processes and improve productivity

  • Emphasis on language models with larger language skills and reasoning abilities

Live Book and Sherpa

  • Development of a ‘live book’ called Sherpa, an interactive book that writes itself

  • Sherpa can be accessed through a companion app and provides on-demand information

  • Sherpa is currently in the Alpha stage and user contributions are encouraged

Community Involvement

  • Collaborative effort involving numerous individuals from the community and company

  • New contributors are welcome to join and have their picture added to the project’s progress slide

Sherpa’s Functionality

  • Sherpa serves as a question-answering agent built on top of a large language model-based pipeline

  • Provides answers based on the content of the open book and indexed documentation

Project Updates

  • Significant progress has been made since the last update

  • Features such as resource citation and improved error handling have been added

  • Ongoing work to enhance multiplayer behavior in the Slack app and improve user experience

Future Plans

  • Focus on expanding the project by adding more tools and resources

  • Indexing up-to-date documentation and creating specialized tools for specific topics

  • Community involvement and contributions are encouraged to enhance project capabilities

Improvements in Sherpa

  • Addressed the issue of taxonomy by fine-tuning a smaller open source model

  • Improved resource citation by providing an ID to the document instead of the actual link

  • Implemented a method to handle links posted in the chat by summarizing the linked page

Availability and Future Plans of SharePoint

  • SharePoint is currently available in public channels

  • Plans to make it available through DMS for more private inquiries

  • Considerations to implement a rate limit for the deployed version

  • Planning to install SharePoint in other communities and make it available on the marketplace

LLM Light: A Journey into Large Language Models

  • Introduction of LLM Light project for exploring large language models

  • Utilization of Colab for finding and working with a relatively large dataset

  • Fine-tuning the dataset involves an auto-regressive approach

  • Options for hosting the model include Hugging Face and Sagemaker Jumpstart

In This Page: