Changyu Lee

NLP & Prompt Engineering

Prompt
Search

Project Overview

Results

3 Analysis of Dataset

Relabeling and Sampling

•
Totally, from the dataset,
Average length before cleaning: 317.2775
Average length after cleaning: 301.1736
•
How I relabeled and sampled

Pre-processing

A Report of Hands-on Project of Text Representations and the Use of Text Classification for Sentiment Analysis
Project
Lecture Summary

Ollama 특정 GPU만 사용하게 하기

•
기본 원리는 간단하다. ollama 서비스 실행 시 CUDA_VISIBLE_DEVICES를 명시해주면 된다.
•
이 글은 linux 기반의 서버를 기준으로 설명한다.
1.
시스템 서비스로 Ollama 관리:
2.
수동으로 Ollama 서버 실행:

이후, CUDA_VISIBLE_DEVICES 옵션 설정

Ollama 서버를 특정 GPU에서 실행하려면 CUDA_VISIBLE_DEVICES 환경 변수를 설정하여 사용할 GPU를 지정할 수 있다. 이렇게 하면 Ollama가 지정된 GPU에서만 실행된다.
•
CUDA_VISIBLE_DEVICES 설정:
•
서비스 파일 리로드 및 Ollama 서버 재시작: 변경 사항을 저장하고 나서, 다음 명령어를 입력하여 시스템 데몬을 리로드하고 Ollama 서버를 재시작.
Ollama 특정 GPU만 사용하기 설정 방법
Log
RAPTOR: Recursive Abstractive Processing for Tree-Organized Retrieval
Retrieval-augmented language models can better adapt to changes in world state and incorporate long-tail knowledge. However, most existing methods retrieve only short contiguous chunks from a retrieval corpus, limiting holistic understanding of the overall document context. We introduce the novel approach of recursively embedding, clustering, and summarizing chunks of text, constructing a tree with differing levels of summarization from the bottom up. At inference time, our RAPTOR model retrieves from this tree, integrating information across lengthy documents at different levels of abstraction. Controlled experiments show that retrieval with recursive summaries offers significant improvements over traditional retrieval-augmented LMs on several tasks. On question-answering tasks that involve complex, multi-step reasoning, we show state-of-the-art results; for example, by coupling RAPTOR retrieval with the use of GPT-4, we can improve the best performance on the QuALITY benchmark by 20% in absolute accuracy.
https://arxiv.org/html/2401.18059v1

Abstract

•
Problems of existing retrieval-augmented approaches
•
selecting the most relevant information for knowledge-intensive tasks is still crucial.

Background

•
Recursive Summarization as Context
•
 long texts often present subtopics and hierarchical structures (Cao & Wang, 2022; Dong et al., 2023b)

Method

Tree Contrection Process

•
RAPTOR recursively clusters chunks of text based on their vector embeddings and generates text summaries of those clusters, constructing a tree from the bottom up. Nodes clustered together are siblings; a parent node contains the text summary of that cluster.
The clustering approach in tree construction includes a few interesting ideas.
GMM (Gaussian Mixture Model)
•
Model the distribution of data points across different clusters
RAPTOR: Recursive Abstractive Processing for Tree-Organized Retrieval
Paper

How Does Vector Similarity Work

Summary by AI:
Vector Similarity Search works by converting data items (such as text documents, images, or audio clips) into vectors in a high-dimensional space. Each vector represents the features or characteristics of its corresponding item in a way that captures semantic or contextual meaning. The process of determining how "similar" two items are involves measuring the distance between their vectors in this space. Let's break down the key components and steps involved in vector similarity search:

1. Vector Representation

•
Text Data: Techniques like Word Embeddings (Word2Vec, GloVe), Sentence Embeddings (BERT, Sentence-BERT), or Document Embeddings are used to convert words, sentences, or entire documents into vectors. These embeddings capture semantic meaning, where similar words or sentences have vectors that are close to each other in the vector space.
•
Non-Text Data: For images, audio, and other non-textual data, deep learning models (such as Convolutional Neural Networks for images) are used to extract features and encode them as vectors.
•
Vector Embedding

2. Similarity Metrics

Once all items are represented as vectors, the similarity between them is calculated using distance or similarity metrics. The choice of metric can depend on the type of vectors and the specific requirements of the application. Common metrics include:
•
Basic Metrics
•
NN Algorithms:
•
Approximate Nearest Neighbor (ANN) Algorithms

3. Indexing and Search Algorithms

To efficiently search through millions or billions of vectors, vector similarity search systems use specialized indexing structures and algorithms. These are designed to speed up the retrieval of items that are most similar to a query vector, often through Approximate Nearest Neighbor (ANN) algorithms. Examples include:
•
Tree-based Indexes: Such as KD-trees and Ball-trees, which partition the vector space into regions.
What is Vector Similarity Search? : 2. How Does Vector Similarity Work
Theory

Vector Similarity Search

Vector Similarity Search refers to the process of finding vectors (arrays of numbers) that are closest to a given query vector in a vector space.
This process is fundamental in various applications such as recommendation systems, search engines, image retrieval, and natural language processing tasks. The idea is to represent data (like text, images, or user preferences) as vectors in a high-dimensional space where the similarity between these data points can be calculated based on their vector representations.
The similarity between vectors is often measured using metrics such as Euclidean distance, cosine similarity, or Manhattan distance, among others. The choice of metric depends on the specific application and the nature of the data.
•
Euclidean Distance measures the straight-line distance between two points in Euclidean space, which is suitable for numerical data.
•
Cosine Similarity measures the cosine of the angle between two vectors, which is particularly useful for text data represented in vector space models (like TF-IDF or word embeddings) because it captures the orientation of the vectors and not the magnitude, making it sensitive to the similarity in direction of the vectors.
•
Manhattan Distance (or L1 distance) measures the sum of the absolute differences of their coordinates, which can be useful for certain types of data distributions or applications where the grid-like distance matters.
Vector Similarity Search can be computationally expensive, especially as the size of the dataset grows. To address this, various indexing and search optimization techniques such as k-d trees, locality-sensitive hashing (LSH), and approximate nearest neighbor (ANN) algorithms are employed to speed up the search process without significantly compromising accuracy.
These techniques enable efficient retrieval of high-dimensional data by approximating the nearest neighbors to a query vector, making Vector Similarity Search a critical component in handling and making sense of large datasets in machine learning and data science.

Prob. to Solve

What is Vector Similarity Search? : 1. Introduction

Concept of Autonomous Agent Sys. [1]

•
Planning
•
Memory
•
Tool use

RAG

•
Retrieval-Augmented Generation
•
Problems of LLM Models
•
RAG is a technique for augmenting LLM knowledge with additional, often private or real-time, data. LLMs can reason about wide-ranging topics, but their knowledge is limited to the public data up to a specific point in time that they were trained on. If you want to build AI applications that can reason about private data or data introduced after a model’s cutoff date, you need to augment the knowledge of the model with the specific information it needs. The process of bringing the appropriate information and inserting it into the model prompt is known as Retrieval Augmented Generation (RAG). [2]

Get Insight from LangChain

What are the Agents? [3]

This is the chain responsible for deciding what step to take next. This is powered by a language model and a prompt. The inputs to this chain are:
1.
Tools: Descriptions of available tools
LLM Agent Overview
Basic

Pydantic

Pydantic은 Python에서 데이터 유효성 검사와 구조화를 위한 라이브러리입니다. Pydantic을 사용하면 데이터 모델을 정의하고 입력 데이터를 검증할 수 있습니다. 이를 통해 코드의 안정성을 높이고 유지 보수를 용이하게 할 수 있습니다.
Pydantic은 간단하고 직관적인 구문을 제공하여 사용하기 쉽습니다. 데이터 모델은 일반적으로 Python 클래스로 정의되며 필드와 필드 유효성 검사 규칙을 포함합니다. 이를 통해 입력 데이터가 예상한 형식과 구조를 따르는지 확인할 수 있습니다.
Pydantic은 다양한 유효성 검사 규칙을 지원하며, 예를 들어 필드의 타입, 최소 및 최대 값, 길이 등을 검사할 수 있습니다. 또한, Pydantic은 JSON 직렬화 및 역직렬화, 데이터 파싱 및 변환 등의 유용한 기능도 제공합니다.
Pydantic은 웹 애플리케이션, 데이터베이스, API 등 다양한 영역에서 사용될 수 있습니다. 문서에서 제공하는 많은 예제와 자세한 설명을 통해 Pydantic의 다양한 기능과 활용 방법을 학습할 수 있습니다.
더 자세한 내용은 Pydantic 문서를 참고할 수 있습니다.

기본 구조

Model

Models are simply classes which inherit from pydantic.BaseModel and define fields as annotated attributes.

Field

The Field function is used to customize and add metadata to fields of models.
Langchain: Pydantic
Basic