Some updates - Fine-tuning of Large Language Moddel

View this file through link Some updates - Fine-tuning of Large Language Moddel (zhengyu.space)

General Idea - What am I doing?

I am trying to use the dataset from MIMIC-III to realise the generation of the "Plan / Follow up" section of the notes from social workers.

Main method:
Fine-tuning Technology for Large Language Models

Research Questions:

1. How can the interview records and basic information of social workers or psychotherapists with patients be effectively used to generate a follow-up plan and provide advice using a chatbot?
2. How do the fine-tuning parameters of large language models affect the quality of generated follow-up plans and the accuracy of advice provided to patients by the chatbot?

Research Aim:

The aim of this study is to develop a chatbot based on a large language model that utilizes interview records and basic information from social workers or psychotherapists with patients to generate personalized follow-up plans and provide effective advice.

Research Objectives:

1. Analyze the characteristics and content of interview records and patient information to determine how to effectively extract and utilize this information for generating follow-up plans.
2. Identify suitable large language models for chatbot development and conduct experiments on fine-tuning parameters to find the most optimal model and parameter combination.
3. Design and develop a chatbot system capable of generating personalized follow-up plans and providing advice based on interview records and patient information.
4. Evaluate the quality of generated follow-up plans and the accuracy of advice provided by the chatbot by comparing them with recommendations from actual social workers or psychotherapists, validating the effectiveness and feasibility of the chatbot.

Key Technology - Fine Tune & LoRA

• have a general idea, no discussion of details
• This section comes partly from Internet sources
  https://en.wikipedia.org/wiki/Fine-tuning_(deep_learning)
  https://zhuanlan.zhihu.com/p/642343404

What is fine-tuning?

Fine-tuning refers to the process of taking a pre-trained model (one that has been trained on a large-scale dataset, like GPT-4 from OpenAI, or Gemma from Google) and then further training it on a specific task or domain of interest. The purpose of fine-tuning is to adapt the broad language comprehension capabilities of these models to more specific tasks or to better reflect the nuances of a particular domain.

Why fine-tuning?

There are several reasons for fine-tuning a language model:

• Specificity: fine-tuning enables the model to understand and generate language for a specific domain or context. For example, a medical AI assistant needs to understand and generate language that is common in the medical context but rare elsewhere.

• Resource efficiency: training a language model from scratch requires significant resources (time, computing power, data). Fine-tuning allows us to take advantage of the learning already done by the model and make it more efficient for our needs.

• Ethical and fairness concerns: large language models can often reflect biases present in their training data. Fine-tuning provides an opportunity to mitigate this bias by controlling the subsequent training process and data.

How to Fine-Tune?

• Define your task and collect data: in order to start fine-tuning, you first need to define the task you want your model to excel at. Then collect relevant data for that task. This could include anything from customer support interaction sets to medical journals, depending on your task.

• Preprocessing and formatting: the next step is to preprocess and format your data. This may include tokenisation (breaking text into smaller parts such as words or sub-words), cleansing (removing irrelevant information), and constructing the data to match the model's expected inputs.

• Training: With the task defined and the data prepared, you are ready for fine-tuning. This involves running the pre-trained model on your data and adjusting its parameters to minimise the difference between its output and the expected output.

• Evaluate and test: After training, it is crucial to evaluate your model. Testing the model on unseen data helps determine how well the fine-tuning process worked. Remember to test on a variety of examples to ensure that your model is robust and generalisable.

• Iterate: Fine-tuning is an iterative process. Depending on the results of your evaluation, you may need to collect more data, adjust your task definition, or tweak the training process.

Dataset for Fine-tune

The structure of fine-tune for different language models is different but not much.

• One possible structure for LoRA
  
• format for models from OpenAI, like GPT-3

{"messages": [{"role": "system", "content": "Marv is a factual chatbot that is also sarcastic."}, {"role": "user", "content": "What's the capital of France?"}, {"role": "assistant", "content": "Paris, as if everyone doesn't know that already."}]}
{"messages": [{"role": "system", "content": "Marv is a factual chatbot that is also sarcastic."}, {"role": "user", "content": "Who wrote 'Romeo and Juliet'?"}, {"role": "assistant", "content": "Oh, just some guy named William Shakespeare. Ever heard of him?"}]}
{"messages": [{"role": "system", "content": "Marv is a factual chatbot that is also sarcastic."}, {"role": "user", "content": "How far is the Moon from Earth?"}, {"role": "assistant", "content": "Around 384,400 kilometers. Give or take a few, like that really matters."}]}

{"prompt": "<prompt text>", "completion": "<ideal generated text>"}
{"prompt": "<prompt text>", "completion": "<ideal generated text>"}
{"prompt": "<prompt text>", "completion": "<ideal generated text>"}

In a nutshell, they are pairs of questions and answers.

About LoRA

Why LoRA？

LLMs are extremely large in size (parameters in the order of billions). Full fine-tuning (which updates all the parameters in the model) is not required for most applications because typical fine-tuning datasets are relatively much smaller than the pre-training datasets.

What is LoRA?

Low Rank Adaptation (LoRA) is a fine-tuning technique which greatly reduces the number of trainable parameters for downstream tasks by freezing the weights of the model and inserting a smaller number of new weights into the model. This makes training with LoRA much faster and more memory-efficient, and produces smaller model weights (a few hundred MBs), all while maintaining the quality of the model outputs.

About the Dataset

The dataset I am using is from MIMIC-III, called "The noteevents table". (link: NOTEEVENTS | MIMIC)

This table contains 2,083,180 rows and is around 3.69GB. Obviously, Microsoft Excel cannot directly open this file. Hopefully, the datasets from MIMIC are also available from Google Cloud and I can directly using SQL (a programming language for database) to access the table.

Using SQL to access the dataset, we can know that column "CATEGORY" has 15 possible values.

I briefly observed that "Social Work" may be the best one to reflect the spiritual needs of patients.
The NOTE from the "Social Work" category almost contains two parts, Patient information and Plan/ Follow up.

Observing the structure of like this, I am considering whether it is able to make something that can generate the output (plan/ follow up) using the input (patient information).

Dataset Preprocessing

Since the output and input that I want is in the same column, I need to separate them into two columns. However, there exist some problems.

1. Some rows don't have the Plan/ Follow up part.
2. Some rows have different representation of Plan/ Follow up, like

• "Plan" (in a separate row)
• "Plan" (followed by text)
• "Plan/ Follow up" (in a separate row)
• "Plan/ Follow up" (followed by text)
• etc.

In my trail so far, I delete all the rows that don't contains "Plan/ Follow up" or the content is empty, and only keep the rows with the format of "Plan/ Follow up" (in a separate row) for my convenient.

After this pre-processing, there are only 576 pieces of data, which is very few indeed.

My trial

Large Language Model choose

Currently, I am trying to use Gemma, one open asscess LLM model from Google.

Although Google says it has a relatively good performance compared with other LLM models, it seems not that good in my experience.

Gemma - a family of lightweight, state-of-the art open models from Google.  |  Google AI for Developers

I have created a notebook in colab to perform fine-tuning Gemma. The screenshot shows that the performance is pretty bad so far. I can say that, GPT-3 or GPT-4 can perform better than fine-tuned Gemma.

Shortcomings

1. the size of the dataset is really small. the dataset I used for fine-tuning only have 576 rows. Generally, doing fine-tuning need at least 10,000 pieces of data.
2. the Gemma model is not that good. There are many other open access LLM models. Google has mentioned some in its coparasion. I may try other models to see if they can perform better.

Following work

1. In the section of “Dataset Preprocessing”, I deleted many potentionally useful data to make my work easier. I may try to delete lesser, and make more data useful.
2. Try other open models. I try Gemma first because it is relatively new. Generally, new things is more attractive.
3. Try models from OpenAI. The models like GPT-3 or GPT-4 are closed access. That means that
   1. it cost money to buy tokens from OpenAI.
   2. the data for fine-tuning would be provided to OpenAI. I am not sure about the privacy protection or the related stuff.

To grind about

1. "playing" with LLM models is relatively expensive. Whether it's purchasing access to an API or purchasing computational units. For example, I spent 78HKD to buy 100 computational units from Google colab to try Gemma, and it run out within 12 hours.
2. I am pretty new with the large language models and not that familiar with them, so I said that I am "playing" with LLM. While I think it is pretty close to one of Dr Vivian's expection, making a chatbot. In my opinion, in line with current AI trends, it is difficult to rely on academia to develop a highly capable generative AI from scratch. It is much more cost-effective to use existing models from some of the strongest companies.
3. I find a interesting website.YU Challenge AI-me
   This website using GPT-3 to built a chatbot for his own competiton to provide some information, just like a online customer service staff.