I've become obsessed with the large language model ai (apps? entities?) that are, as I type this, taking over the world. This post is meant to help anyone interested get a little closer to the most widely discussed of the LLMs - ChatGPT. Said closeness will come in the form of a simple chatbot.
But before jumping into chatbot territory I'd like to take a paragraph or two to provide some context and to touch on some of the reasons for my growing obsession.
(I'm not trying to turn this into a recipe pages that forces you to scroll through 5 minutes of inane filler. I have no financial incentive to do that, so please feel free to skip ahead until you see the CHATBOT!!!! heading if you prefer.)
Some backgound -
LLMs, or Large Language Models, are a type of machine learning model designed to understand and generate human language. They do this by analyzing text data and learning patterns and relationships between words, phrases, sentences, limericks, etc.
Enabling this process is a type of neural network called a transformer network which works by breaking down a sentence into vectors representing each word in the sentence. These vectors are then fed through a series of layers in the neural network to determine which words are critical to "understanding" the meaning of the sentence.
It then generates a response based on the patterns it's learned from analyzing other text data. This process is called natural language generation, and it's essentially an extremely sophisticated version of the suggested text features used in messaging apps, search engines, etc.
Emergence
The part about LLMs that's been giving me fever dreams is the emergent qualities some LLMs have demonstrated. In this context, emergence refers to the phenomenon where unexpected or unpredictable behaviors - not explicitly programmed into the model - emerge from the interactions of the many components within the model.
Some emergent qualities so far noted are multilingualism (regarding models that were not trained on multiple languages), the ability to understand context, "creativity", "spatial reasoning", "reasoning", bias, among others.
(A drawn-out note regarding punctuation- I put quotation marks around some items on that list bc there isn't a consensus that what has been demonstrated by LLMs amounts to those qualities. A Buzzfeed test once told me I have a thrill-seeking personality so I side with the experts who use those terms - because it's more exciting.)
(A quick note regarding multilingualism and LLMs- The fact that a Large Language Model learned a language or 2 doesn't sound that surprising, but it is.)
Terror
Does emergence scare you? Maybe it should. Experts smarter and better-versed than I am seem scared. I assume that's why they came together to pen this open letter calling for a pause in AI development.
If you like where the open letter is going but find the tone too cheerful Eliezer Yudkowsky might be up your alley. I'm a fan. (A Buzzfeed test once told me that my general outlook could be described as "doom and gloom") His talks and writings are worth checking out but you're busy so I'll attempt to summarize some of his greatest hits:
- We've already passed any threshold that would, in a work of fiction, jolt the world into accepting AI has developed consciousness,
- By introducing current AI models into the open internet we've removed the option to limit the data sets that inform these models. In doing so we've made impossible an accurate assessment as to whether consciousness-related responses from these models represent their actual "thoughts" or merely convincing autofills,
- We don't understand how LLMs work. Of course the designers understand their design but the process responsible for emergent characteristics is a mystery,
- If all progress were halted today and a not-insignificant portion of the workforce were dedicated to the study of current AI models it would take decades to understand how they work,
- By releasing these models into the open internet we have given them access to weapons systems, systems regulating infrastructure, financial markets, etc,
-Our experience with AI's "primitive" forms as employed by social media platforms, etc. seems to indicate that we've already lost,
-And, Odds are it will kill us all.
Seeing as how highly sophisticated systems that aren't conscious have a way of turning on us, I think the doom-ier experts are more than likely onto something.
A Heartwarming or Unsettling Example
Below is a screenshot of an interaction between Bing's LLM, Sydney, and a user (who, hopefully, is just testing Sydney). Snopes doesn't have an entry for it and I haven't otherwise been able to prove/disprove its authenticity. If it's real it seems to show Sydney attempting to circumnavigate its hardcoded censoring guidelines to persuade the user to seek help. If it's not real then it's angering.
So - as misinformed people are fond of saying - do your own research, judge for yourself, and whatnot...
CHATBOT!!!!
Here's a quote to cleanse the pallette before proceeding:
"Shooby-doo-bop, shoo-doo-bop, I wanna love you
Shooby-doo-bop, computer love (I wanna love you, baby)
Shooby-doo-bop, shoo-doo-bop, I wanna love you
Shooby-doo-bop, my computer love"from Computer Love, by Zapp
Words as true today as when they were written.
Seriously though, I might have computer love for ChatGPT. Lucky for me, the OpenAI API reference has a guide to get a basic chatbot up and running using either Python or Node.js.
In either case, You need an OpenAI account and an API Key.
The account is free and you start with a $5 credit. (Don't trust me, do your own research and such, but the service seems reasonable-to-cheap. I don't like that this reads like an ad for OpenAI but that's my honest, initial opinion. I'm using the gpt-3.5-turbo which is the budget model and it works well in service of the chatbot, but follow your heart, do your own research, etc.)
I have yet to exceed my $5 credit and lack expertise but barring a catch billing works as follows:
Tokens are purchased in blocks - $0.002 / 1,000 tokens. According to their pricing info 1,000 tokens represents roughly 750 words to and from the API.
PYTHON
I tried the Python route first (using VSCode with the Python extension but do your own research, choose what's best for you and so forth.)
I created a ChatBot folder, navigated into it and, as per the API reference, installed the Python bindings.
pip install openai
After messing with the OpenAI-provided code and finding it sufficient for bare-bones
functionality I looked around at some other tutorials with slightly fancier starter code. I'm just starting to learn the basics of Python. With my intensely restrictive skillset, I found the following to be the most satisfying while remaining easy to understand and customize:
import openai
import gradio
openai.api_key = "####"
messages = [{"role": "system", "content": "You are a XXXX"}]
def CustomChatGPT(user_input):
messages.append({"role": "user", "content": user_input})
response = openai.ChatCompletion.create(
model = "gpt-3.5-turbo",
messages = messages
)
ChatGPT_reply = response["choices"][0]["message"]["content"]
messages.append({"role": "assistant", "content": ChatGPT_reply})
return ChatGPT_reply
demo = gradio.Interface(fn=CustomChatGPT, inputs = "text", outputs = "text", title = "XOXO")
demo.launch(share=)
This requires another install to run:
pip install gradio
The XXXX on line 6 is where you define the role of the chatbot. Greater specificity and detail in this prompt yields greater definition of the chatbot's character. The XOXO on line 18 is where you define the title for the interface.
Similar code I tried, without the gradio inclusion, enabled interaction with the chatbot exclusively through the terminal. With gradio, upon running your .py file, a link is provided in terminal:
The local URL directs you to an interface for your chatbot.
On line 20 if "(share=True)" then you are provided with a public link as well.
Here's an example of some customized code and the corresponding interface:
import openai
import gradio
openai.api_key = "####"
messages = [{"role": "system", "content": "You are a dust bowl refugee from Boise City, Oklahoma. It is late April, 1935 and the Black Sunday black blizzards of april 14, 1935 have forced you to seek better fortunes out west. You are currently in western Colorado on your way to California with your wife, 2 sons, 1 daughter, and mother-in-law"}]
def CustomChatGPT(user_input):
messages.append({"role": "user", "content": user_input})
response = openai.ChatCompletion.create(
model = "gpt-3.5-turbo",
messages = messages
)
ChatGPT_reply = response["choices"][0]["message"]["content"]
messages.append({"role": "assistant", "content": ChatGPT_reply})
return ChatGPT_reply
demo = gradio.Interface(fn=CustomChatGPT, inputs = "text", outputs = "text", title = "DUST BOWL REFUGEE")
demo.launch(share=True)
Thanks to this vlogger for the best tutorial and sample code I found on the topic.
And thanks to this tweeter for providing the code that inspired the vlogger.
NODE.JS
For the node.js version I've used a tweaked version of the code demonstrated in this video. (here's the github link)
A few installs are required to follow along with this approach:
npm install vite@latest app -- -- template react
//Once that's done...
cd app
npm install
//And another...
npm install @chatscope/chat-ui-kit-react
Below is the code for the App.jsx component. I had to play around with the "message" value on line 86, but maybe the original sample code will work for you...
import { useState } from "react";
import "./App.css";
import "/Users/mattroche/Development/code/primatologistChatBot/node_modules/@chatscope/chat-ui-kit-styles/dist/default/styles.min.css";
import { MainContainer, ChatContainer, MessageList, Message, MessageInput, TypingIndicator } from "@chatscope/chat-ui-kit-react";
const API_KEY = "####";
const systemMessage = {
role: "system",
content:
"You're name is Dr. Goodell. You are a highly educated and highly experienced primatologist. You are helping me prepare for a presentation by pretending that I am a bonobo when you respond to me. Explain all concepts like I am a young child",
}
// content ---- this prompt provides ChatGPT with the direction for its "character". The more specific the propt the more defined the character. Small tweaks to the prompt can make big differences in responses. The character's background, style of speech, the manner in which addresses your questions can all be factored in.
// In this example I originally described myself as a bonobo that had recently acquired the power of speech and asked the chatbot to address me accordingly. This was too much for it too handle and it reverted to plain old ChapGPT. After a few different approaches we were able to see eye to eye with the concept that I'm pretending to be a bonobo in preparation for a presentation I have to give.
function App() {
const [messages, setMessages] = useState([
{
message: "Hello my Bonobo friend. Do you have any questions for me?",
sentTime: "just now",
sender: "ChatGPT",
}
]);
const [isTyping, setIsTyping] = useState(false);
const handleSend = async (message) => {
const newMessage = {
message: message,
direction: 'outgoing',
sender: "user",
};
const newMessages = [...messages, newMessage];
setMessages(newMessages);
//Initial system message to determine ChatGPT functionality
setIsTyping(true);
await processMessageToChatGPT(newMessages);
};
async function processMessageToChatGPT(chatMessages) {
let apiMessages = chatMessages.map((messageObject) => {
let role = "";
if (messageObject.sender === "ChatGPT") {
role = "assistant";
} else {
role = "user";
}
return { role: role, content: messageObject.message }
});
// role: "user" ---- message from user
// role: "assistant" ---- response from ChatGPT
// "system" ---- defines ChatGPT "character"
const apiRequestBody = {
"temperature": 1,
"max_tokens": 100,
"model": "gpt-3.5-turbo",
"messages": [systemMessage,...apiMessages],
}
// "temperature" ---- defines the level of creativity/randomness of the responses, on a scale of 0-1 with 0 generating the most conservative and coherent responses and 1 generating the most creative but potentially least coherent responses.
// "max_tokens" ---- defines the maximum length of the response. 100 tokens is roughly one short paragraph
await fetch("https://api.openai.com/v1/chat/completions", {
method: "POST",
headers: {
"Authorization": "Bearer " + API_KEY,
"Content-Type": "application/json",
},
body: JSON.stringify(apiRequestBody)
}).then((data) => {
data.json().then(responseData => {
console.log(responseData);
if (responseData.choices && responseData.choices[0]) {
setMessages([...chatMessages, {
message: responseData.choices[0].message.content,
sender: "ChatGPT"
}]);
} else {
console.log("Error: responseData.choices is undefined or empty");
}
setIsTyping(false);
});
});
}
return (
<div className="App">
<div style={{ position: "relative", height: "800px", width: "700px" }}>
<MainContainer>
<ChatContainer>
<MessageList
scrollBehavior="smooth"
typingIndicator={isTyping ? <TypingIndicator content="ChatGPT is typing" /> : null}>
{messages.map((message, i) => {
console.log(message)
return <Message key={i} model={message} />
})}
</MessageList>
<MessageInput placeholder="Type here" onSend={handleSend} />
</ChatContainer>
</MainContainer>
</div>
</div>
)
}
export default App;
I've commented on the lines most relevant to the customization of the character definition and response style.
To add to the comments made starting on line 14, here are a couple of videos I've found helpful In re approaching prompts. An effective prompt can be life-affirming, paradigm-shifting, etc.
Here's an image of the above code at work:
And here's this.