[AINews] Evals-based AI Engineering • ButtondownTwitterTwitter

The AI Reddit Recap section provides a detailed overview of new models, architectures, quantization techniques, stable diffusion enhancements, humor, and memes discussed across various AI-related subreddits. It covers topics such as new AI models like Jamba and Bamboo, Quantization advancements like 1-bit Llama2-7B and QLLM, Stable Diffusion Enhancements like Hybrid Upscaler Workflow and IPAdapter V2, and humorous content like AI Lion Meme and 'Filming animals at the zoo'.

Quantum Leaps in Hugging Face Contributions:

New advancements have been made in AI research and applications: HyperGraph Representation Learning provides novel insights into data structures, Perturbed-Attention Guidance (PAG) boosts diffusion model performance, and the Vision Transformer model is adapted for medical imaging applications. The HyperGraph paper is discussed on Hugging Face, while PAG's project details are on its project page and the Vision Transformer details on Hugging Face space.

Colab and Coding Mettle: Engineers have been sharing tools and tips ranging from the use of Colab Pro to run large language models to the HF professional coder assistant for improving coding. Another shared their experience with AutoTrain, posting a link to their model.

Model Generation Woes and Image Classifier Queries: Some are facing challenges with models generating infinite text, prompting suggestions to use repetition penalty and StopCriterion. Others are seeking advice on fine-tuning a zero-shot image classifier, sharing issues and soliciting expertise in channels like #NLP and #computer-vision.

Community Learning Announcements: The reading-group channel's next meeting has a confirmed date, strengthening community collaboration. Interested parties can find the Discord invite link to participate in the group discussion.

Real-Time Diffusion Innovations: Marigold's depth estimation pipeline for diffusion models now includes a LCM function, and an improvement allows real-time image transitions at 30fps for 800x800 resolution. Questions on the labmlai diffusion repository indicate ongoing interest in optimizing these models.

• International Shipping Hacks: Workarounds for international delivery of the O1 light include buying through US contacts. O1 devices built by users are functional globally.
• Local LLMs Cut API Costs: Using Open Interpreter in offline mode eliminates API costs. Running it with local models like LM Studio involves specific commands.
• Collaboration on Semantic Search: Call for enhancing local semantic search within the OpenInterpreter/aifs GitHub repository. Focus on community-driven improvement.
• Integrating O1 Light with Arduino: Discussions on merging O1 Light with Arduino hardware for enhanced utility. Interest in experimenting with alternatives like Elegoo boards.
• O1 Dev Environment Installation Woes: Members report and discuss issues with installing the O1 OS on Windows systems. GitHub pull request aims to streamline the setup for Windows-based developers.

The general-chat section delves into various AI discussions, including the potential of the SD3 Turbo model, concerns about AI art and politics, speculations on LoRA models training on SD3, and the feasibility of training smaller LoRA models on 16GB hardware. Members also recommend leveraging new models like Arc2Face for face manipulation. The section showcases an active exchange of ideas, techniques, and optimization methods for AI training.

Users in the LM Studio channel expressed excitement for new features like the Branching system and highlighted the usefulness of having folders for branched chats. Some users also requested a 'Delete All' feature for a smoother user experience. The community in this channel shares helpful conversations, YouTube guides, and GitHub links to support each other in troubleshooting technical challenges and exploring the capabilities of different models.

This section highlights specific discussions and activities in various LM Studio channels. It includes members' experiences with AI tools, such as VRAM inaccuracies and seeking stable models amid large volumes of data. Discussions on power supply recommendations, GPU power limits, and leveraging legacy hardware are also covered. Additionally, members explore the utility of NVLink for model inference and discuss compatibility queries. The section also delves into user proposals for GPU/NPU monitoring, troubleshooting GPU offloading issues, and stability questions regarding ROCm Beta releases. Furthermore, it touches on integrating LM Studio into Nix, sharing GitHub PR links, and inquiries about JSON outputs from LM Studio.

In this section, discrepancies in weight parameters between Transformer Lens and Hugging Face versions of models like GPT-2 and Pythia 70m were explored, revealing variations that were initially thought to be a bug but were later confirmed as a feature by Transformer Lens. The section also delves into the comparison of same-shaped logits, highlighting their significant absolute differences before applying softmax, which led to consistent relative order and output. Additionally, the process of identifying weight disparities involved plotting matrices and analyzing characteristics. Another discussion focused on the impact of reconstruction errors in Sparse Autoencoders (SAEs), emphasizing their potential to significantly alter model predictions more than random errors of similar intensity.

Structured Inputs for RAG

• Proposed using structured formats like XML for input delineation in modeling.
• A dedicated category for pydantic related implementations is suggested, indicating a trend towards more structured and metadata-rich inputs for AI models.

Updates have been made to several Mojo packages such as mojo-prefix-sum, mojo-flx, mojo-fast-base64, and mojo-csv to version 24.2. Mojo hash and compact-dict are partially updated with ongoing issues. A community member expressed interest in learning about Mojo's underlying MLIR dialects to contribute directly. Information about MLIR in Mojo is expected to undergo significant changes. Mojo's internal dialects will be made more available over time with the current focus on stabilizing MLIR syntax. The Reference feature in the Mojo language is evolving and is expected to continue improving.

• Emphasized the utilization of Text Mining models and BERT for a PII detection project.
• Discussed various topics under different HuggingFace channels such as computer vision and NLP.
• Mentioned incidents related to SAM model fine-tuning troubles and YOLO model conversion.
• Highlighted challenges faced by users in image classifier fine-tuning and Pix2Pix prompt testing.
• Shared insights about issues in BART CNN model and seeking research collaborations in NLP community.
• A member inquired about the date of the next meeting within the reading-group.

OpenInterpreter Discussions:

• O1 Light International Delivery Options: Discussions on ordering the O1 Light for international delivery through US friends.
• Running Open Interpreter in Offline Mode: Users exploring offline mode usage to avoid API costs and discussing local providers like Jan, Ollama, and LM Studio.
• Contribution Invites to OpenInterpreter Projects: Call for contributions to the OpenInterpreter/aifs GitHub repository.
• AI in Industrial Design: Reminiscing about early voice-activated AI companions in industrial design like Jerome Olivet's ALO concept phone.
• Debugging O1 Projects in Coding IDEs: Conversation on debugging O1 projects using PyCharm, Visual Studio Code, and local model servers like LM Studio.

AI21 Labs introduces Jamba, a state-of-the-art model with a novel architecture merging elements of Mamba and the Transformer. It features a 256K context window, the ability to fit up to 140K context on a single GPU, and delivers an unprecedented 3X throughput for use cases like question/answering and summarization. Jamba is now available with open weights under the Apache 2.0 license, accessible on Hugging Face, and soon to be included in the NVIDIA API catalog. The TechCrunch Exclusive discusses Jamba's competition with other models like OpenAI's ChatGPT and Google's Gemini, highlighting its large context windows that are less compute-intensive.

This section delves into detailed discussions about various advanced AI models. It covers topics such as the introduction and technical specifications of new models like Jamba and Qwen1.5-MoE, as well as debates around model architectures and efficiency. Links to GitHub repositories and Hugging Face spaces are provided for further exploration and experimentation.

• Token Insertion Troubleshooting: A member faced unexpected token insertions attributed to quantization or the engine used, resolved by supplying an added_tokens.json file with specified tokens.
• Translation Model Showdown: Interest in comparing translation outputs from models like DiscoLM, Occiglot, Mixtral, GPT-4, DeepL, and Azure Translate. The idea involves translating the first 100 lines of a dataset like Capybara through each service.