In today’s MindBytes:

• Chinese tech giant Alibaba to roll out ChatGPT rival

• IDEA Research releases Grounded-Segment-Anything

• Generative Agents: Interactive Simulacra of Human Behavior

• TurboPilot, a self-hosted GitHub Copilot

Read time: 2 minutes

Was this email forwarded to you? Make sure you subscribe for more!

Chinese tech giant Alibaba to roll out ChatGPT rival

Chinese technology giant Alibaba has announced plans to roll out its own artificial intelligence (AI) ChatGPT-style product called Tongyi Qianwen. Capable of working in English and Chinese, it will initially be added to DingTalk, Alibaba's workplace messaging app, performing tasks like transcribing conversations, writing emails, and drafting business proposals. Alibaba also plans to integrate the chatbot into Tmall Genie, a smart speaker similar to Amazon's Alexa.

Source: https://www.bbc.com/news/business-65236848

IDEA Research releases Grounded-Segment-Anything

The core idea behind this project is to combine the strengths of different models in order to build a very powerful pipeline for solving complex problems.

• Segment Anything is a strong segmentation model. But it needs prompts (like boxes/points) to generate masks.

• Grounding DINO is a strong zero-shot detector which is capable of to generate high quality boxes and labels with free-form text.

• The combination of Grounding DINO + SAM enable to detect and segment everything at any levels with text inputs!

• The combination of BLIP + Grounding DINO + SAM for automatic labeling system!

• The combination of Grounding DINO + SAM + Stable-diffusion for data-factory, generating new data!

• The combination of Whisper + Grounding DINO + SAM to detect and segment anything with speech!

Grounded-SAM + Stable-Diffusion Inpainting: Data-Factory, Generating New Data!

BLIP + Grounded-SAM: Automatic Label System!

Using BLIP to generate caption, extract tags and using Grounded-SAM for box and mask generating. Here's the demo output:

Generative Agents: Interactive Simulacra of Human Behavior

Paper: https://arxiv.org/pdf/2304.03442.pdf

Demo: https://reverie.herokuapp.com/arXiv_Demo/

• What are Generative Agents? Computational software agents that simulate believable human behavior in interactive environments.

• What Can They Do? Perform everyday activities (e.g., cooking, working), engage in hobbies (e.g., painting, writing), form opinions, interact with others, remember past experiences.

• How Do They Work? Architecture uses a large language model to:

  • Store experiences in natural language

  • Synthesize memories into higher-level reflections

  • Retrieve memories for planning behavior

• Interactive Sandbox Environment: Inspired by The Sims, users interact with a small town of 25 agents using natural language

• Evaluation Results: Agents demonstrated believable behaviors, such as spreading invitations to a Valentine's Day party and coordinating attendance.

• Potential Impact: Combining large language models with interactive agents to create simulations of human behavior.

TurboPilot, a self-hosted GitHub Copilot

TurboPilot is a self-hosted copilot clone which uses the library behind llama.cpp to run the 6 Billion Parameter Salesforce Codegen model in 4GiB of RAM. It is heavily based and inspired by on the FauxPilot project.