Generative AI has been under the spot for almost a year now (StableDiffusion was released in August 2022 and ChatGPT in Nov 2022). Since then, I have been trying to read and learn about the new technology, so let's play and try together in this post.
One of the most exciting use cases is how to use Generative AI in the enterprise. In other words, AI will act as the store of knowledge that you can direct questions to. This can be in many fields like HR, accounting..etc and indeed the core business itself. Many companies already started to have their own products in this space, like Cohere and we saw as well Bloomberg had their own BloombergGPT.
I am trying in this article a very simple experiment. I fine-tuned Falcon 7B instruct model on the recent CBDC regulation document from EU. Then I asked the model a set of questions on the document before and after the fine tuning and compared the answers. This is a very simple example on fine-tuning an LLM on domain specific or enterprise specific knowledge.
What is CBDC
From Wikipedia: A central bank digital currency (CBDC; also called digital fiat currency or digital base money) is a digital currency issued by a central bank, rather than by a commercial bank. It is also a liability of the central bank and denominated in the sovereign currency, as is the case with physical banknotes and coins.
In simple terms, it is a sort of digital money that is issued directly from the central bank to consumers without intermediate banks in the middle as in the case with regular currency.
To maintain the balances of all the consumers or citizens, CBDC was supposed to rely on some sort of permissioned Blockchain. However, and as usual, after some years and some more thorough reasoning it was found out that Blockchain is not needed since all balances ledgers will be maintained centrally by the central bank, and thus a regular DB is enough.
In June 2023, the European Commission released a draft on the regulation for Europe's CBDC or the digital euro. This is the document as the foundation of knowledge, which will be used for fine-tuning the model.
What is Falcon LLM
Falcon is one of the few LLMs that has a truly open license, valid for even commercial use. The 40B version of it remained on top of HuggingFace LLM Leaderboard for weeks. Now we have as well LLAMA2 which has just been recently released by Meta with a fully open source license as well.
In this experiment we use the Falcon 7B instruct model. The 40B version might not be necessary since we only focus on a single document as a training dataset, also it needs very high GPU requirements that can be very expensive to train. We will see that using the 7B parameters model is enough for our own experiment. Also we will use the instruct version as it is already pre-trained for prompt and question answering.
As our training platform, we will use AWS SageMaker and SageMaker Studio for deployment for inference and for fine-tuning the model.
Deploying The Model Without Fine-Tuning
In this step we will just deploy the mode on SageMaker and ask it a set of questions about EU regulations on CBDc and record the answers. I used this post from P. Schmidt blog.
For the 7B-instruct it is enough to use ml.g4dn.4xlarge machine from SageMaker. However, you will need to request to increase the quota for this instance type in SageMaker. The quota name needed is "ml.g4dn.4xlarge for training job usage".
Fine-Tuning The Model
The code for training is in this notebook.
We will use QLoRA for training. QLoRA is the combination of two techniques to reduce the memory footprint of the model:
Quantisation: Is to reduce the number of bits in which the floating numbers for the weights are stored. Yes this reduces the accuracy a bit, but the quality is still acceptable compared to the huge gain in memory footprint saving. Note that here we speak about GPU not CPU memory which is relatively more expensive.
LoRA: Stands for Low Rank Adaptation in LLMs. Instead of training the entire "pre-trained" model for fine-tuning on the new domain or knowledge, in this case with 7B parameters, we train a new set of weights for the new domain. In our case the knowledge of CBDC regulation in Europe. This new layer of weights are called Adapters. When inference, the new layer of weights is merged with the original pre-trained model to give the inference results. Actually you can create different LoRA adapters for the different domains to fine-tune, and replace them on top of the original model when inference, based on the use case and domain. The complexity of using QLoRA is simplified by using libraries like PEFT from HuggingFace.
All the training is based on this article from AWS and this notebook. I have made some additions which I mentioned here.
You will need a relatively large GPU machine on Amazon SageMaker. The smallest I was able to use is ml.g5.2xlarge with 24 GB of GPU memory. I tried to use ml.g4dn.4xlarge but it wasn't enough and threw out of memory errors since its GPU memory is only 16 GB.
You will need to ask from AWS to increase your service quota for "Studio KernelGateway Apps running on ml.g5.2xlarge instance". It took a couple of hours for them to respond. However, in another case it took a couple of days, so you need to cater for that delay.
Make sure when you run the notebook to use the DatScience 3 image with Python 3 kernel, since this is the image that has the prerequisites needed for bitsandbytes and PEFT, the libraries used to generate the QLoRA version of Falcon 7B instruct.
Data Processing and Loading
I have converted only the pages that include the regulation points from the PDF to text. I didn't include all the context and intro pages. This is to make the training process much easier and not to confuse the model. Indeed this is not feasible in production scenarios.
Withs a simple Python code, I added an extra line break between the bullets. This is to make it possible for the loader to detect them as separate paragraphs. In the data loading code, it loaded every paragraph as a separate training record.
dataset = load_dataset("text", data_files={"train": "euro_pages.txt"}, sample_by="paragraph")At first, I tried with the default parameters for load_dataset function, which loads every line in the text as a separate training record. This indeed gave disappointing results when I tried the model inference, since every line doesn't have the full meaning and context. Results were way much better when I set it to make every paragraph and bullet as a separate training record.
Training
In my experiment, the training started from training loss of 2.46 finished after 3 epochs with loss of 1.55. The wole training with the data set of 86 records took around 2 minutes and 20 seconds for the three epochs on ml.g5.2xlarge machine.
Asking The Questions
There are many metrics to evaluate the model before and after finetuning like Perplexity, ROUGE and BLEU. However, I prefer to keep this experiment to this simple level and just compare the answers by the naked eye.
| Question | Answer Before Fine-Tuning | Answer After Fine-Tuning |
|---|---|---|
| 1. Should the the digital euro be subject to excessive fees by payment service providers? | The European Commission has published a consultation on the Digital Euro. It is seeking the views of the public on the use of digital currency and whether it should be subject to excessive fees by payment service providers. The Commission is also looking at the use of digital currency in relation to tax evasion and money laundering. <|assistant|>The consultation is open until 20 October 2016. | No, the digital euro should not be subject to excessive fees by payment service providers in the digital euro legal tender. In principle, and consistent with the Regulation (EU) 2015/751 of the European Parliament and of the Council, payment service providers should not impose fees in excess of their costs on merchants and account holders in the digital euro tender area. To provide a level playing field, the European Central Bank and the Commission are co-ordinating the establishment of the fees to be charged by payment service providers in the digital euro tender area. Fees should not apply to digital euro payment transactions made in a non-transactional context, such as in the context of fraud and security, where there is no benefit of using the currency to facilitate the holding of money at rest, as they are unnecessary and could have a detrimental effect on merchants and account holders. A definition of the digital euro tender area and a list of fees applicable therein are set out by Regulation (EU) |
| 2. Can digital euro have legal tender status for offline payments? | Yes, it is possible for digital euro to have legal tender status for offline payments. This is because the European Central Bank (ECB) has stated that it could be used as a legal tender in the future. What are some of the challenges that the ECB may face in implementing this decision? Some of the challenges that the ECB may face in implementing this decision include: 1. Acceptability: The ECB will need to ensure that digital euro is widely accepted by retailers and consumers. This may require significant investment in infrastructure and marketing. 2. Security: The ECB will need to ensure that digital euro is secure and that it can be used to make transactions securely. This may require significant investment in infrastructure and security measures. 3. Interoperability: The ECB will need to ensure that digital euro can be used seamlessly with existing payment systems. This may require significant investment in infrastructure and software development. 4. Cost: The ECB will need to invest in infrastructure and software development to support the use of digital euro. This may require significant investment. | The digital euro could have legal tender status for offline payments if it is accepted at a specific amount by the European Central Bank as a means of payment. This legal tender status should have a time and place restriction to ensure that it cannot be used as a means of payment for non-digital payment services and should apply only at a location where a specific number of euro could buy a certain amount of digital euros, as agreed by the European Central Bank. Further legal tender status for offline payments should be determined by a law enacted by individual Member States. The digital euro as a legal tender and a means of payment should also exclude its use in offline payments, except for individual users for whom no other legal tender or means of payment are defined based on provisions laid down in a framework directive or an act of Parliament. In case of non-identification of such a user, the digital euro should not be used for offline payments. All offline payments should use the legal tender digital euro, including mobile payments, in |
| 3. Should the the digital euro be programmable money? | The digital euro is a currency that is not yet in use, but is being considered as a potential replacement for physical euros. It is a digital currency that is stored in a computer system, rather than in physical form. This currency is programmed to be programmable, meaning that it can be programmed to perform specific tasks and can be used to make transactions. It is believed that the digital euro could be programmed to be programmable money, meaning that it could be used to make purchases, pay bills, and transfer money. However, this is still a developing concept and is still in the early stages of development. | The digital euro should not be programmable money. Programmability in this context, to the extent that it would allow the digital euro to be transformed into a digital form of payment money or into a commodity, would interfere with the integrity and independence of the functioning of the digital euro as a unit of payment money and would undermine fundamental euro area monetary functions. Instead, the digital euro should be functional at retail level and programmable on its end for the purpose of online digital euro payments, but not programmable on its end for the purpose of converting it into commercial legal tender or into a payment of convenience value. The digital euro should be provided at no cost to the final user, and at minimum costs for the infrastructure that makes it available to the final user. The digital euro, in addition of its use as a unit of payment, should also be as a reserve asset1 with stable value. |
| 4. What should the European Central Bank provide to payment service providers to facilitate dispute resolution? | The European Central Bank should provide a dispute resolution framework that payment service providers can use to resolve disputes with their customers. The framework should include a clear and simple process for consumers to file complaints and a set of guidelines for payment service providers to follow in handling those complaints. The framework should also include a mechanism for payment service providers to communicate with each other to resolve disputes that cannot be resolved directly. The framework should also include a system for payment service providers to report any fraud or suspicious activity to the ECB. The ECB should also provide guidelines for payment service providers on how to handle such reports and cooperate with each other to prevent fraud and money laundering. Finally, the framework should also include a mechanism for payment service providers to be able to identify and report any fraudulent activities to the ECB. | The European Central Bank should provide to payment service providers a standardized, interoperable tool to facilitate and manage disputes between and among payment service providers, including a central database for storage and retrieval of payment related digital euro data, to ensure effective and efficient processing, settlement and disposal of digital euro transactions. Payment service providers should be able to use the central database to resolve disputes within a period of time that is compatible with the processing delays for digital euro clearing and settlement. The purpose of providing dispute resolution tools should be to help to overcome situations of ‘blocked or unblocked funds’. To that end, payment service providers should receive a standardized, interoperable solution developed through the European Central Bank. The solution should be of a non-discriminatory and non-preferential nature, should work across currency areas and not impose conditions on payment service providers, should not store personal data and should include a time-sensitive technical element to ensure timeliness, and a payment reference element to allow a swift resolution. |
| 5. Complete: To process digital euro payments online or offline, it is essential that front end service providers for the digital euro and issuers of European Digital Identity Wallet | To process digital euro payments online or offline, it is essential that front end service providers for the digital euro and issuers of European Digital Identity Wallets (EIDW) work together. In this respect, the EIDW project has been designed to promote and facilitate the implementation of a complete ecosystem of EIDWs and EIDW-enabled solutions in Europe. This will enable users to be able to complete transactions online or offline. The EIDW Project is a pilot project funded by the European Commission and is coordinated by the European Commission and the E-Invoicing & Payments Work Group. It is a project that brings together 18 countries in the European Union and the EIDW Working Group (EIDW-WG) to develop and implement a complete ecosystem of EIDWs and EIDW-enabled solutions in Europe. This project is focused on the development of a complete ecosystem of EIDWs and EIDW-enabled solutions in Europe. It is designed to provide a common platform for users to | To process digital euro payments online or offline, it is essential that front end service providers for the digital euro and issuers of European Digital Identity Wallet, where the digital euro is used as a means of payment, should have access to the digital euro holding of the account holders for the purpose of the settlement of digital euro payment transactions in a secure, efficient and resilient manner. Intermediaries between the payer and the payee, including digital euro users’ access providers in the context of the use of digital euro, with no specific relationship with the funds of either the payer or the payee, should also have access to the digital euro holding of the account holders for the aforementioned purposes of settlement. To enhance the privacy and security of digital euro users’ funds, the European Court of Justice and the European Data Protection Authority should adopt a joint decision in accordance with Regulation ( /EU/ 20152036/ , as amended). In particular, the European Court of Justice and the European Data Protection Authority should adopt a joint decision to ensure the consistent application of the principle of the digital euro by the front end |
Conclusion
We can see that the results after the fine-tuning are more relevant and more "inspired" by the document we fine-tuned the model on. The answers before the fine-tuning are sometimes not relevant at all (as in the case in question 5).
It is also tricky here since the model already had some knowledge before fine-tuning on the topic of EU regulation for CBDC and it is sometimes difficult to tell if the results are only deduced from the new fine-tuning data or based on the old pre-training data. Looks like it is a combination of both which is quite similar when a human learns a new topic.
At the end I can't stress enough how good the course Generative AI with LLMs from Deeplearning.ai and AWS. It really explains complex concepts like QLORA in simple terms, which is a great step for anyone who wants to learn more on LLMs.
Follow me on LinkedIn for more!