Wednesday, December 28, 2022

The Pros and Cons of Relying on GPT - day-to-day

Boosting Productivity but Atrophying Problem-Solving Skills?

As someone who frequently uses GPT (Generative Pre-trained Transformer) technology, I've experienced firsthand the benefits it can bring to my work.

But as I've become more reliant on GPT, I've also started to notice a downside: a loss of confidence in my own problem-solving skills.

While this is just one of many potential risks and costs associated with using GPT, it's a concern that I've personally experienced and want to explore further in this article.

Similarly, the benefits of GPT that I'll discuss in this article are just a few examples of the many benefits that GPT can bring.

The Benefits of Using GPT

One of the main benefits of GPT is its speed. Because it can take care of a lot of the "grunt work," I'm now often left free to refocus my efforts on other parts of a problem while it's doing it's thing.

This not only saves time, but it also helps me stay focused on the task at hand.

Additionally, because I'm not personally invested in each strategy, I'm more likely to let go of ideas that aren't working and try something new.  This opens up the possibility of thinking outside the box and considering approaches I may have never considered before. These are just a few examples of the many benefits of using GPT.

An Already Emerging Costs of Relying on GPT

The downside to using GPT is that it can lead to a loss of confidence in my own problem-solving skills.

Now, when I unexpected lose access to GPT (which still happens frequently), I suddenly feel like I can't do anything on my own.

Even when I know I'm perfectly capable of solving a problem independently, I finding that I literally have to intentionally remind myself that I still have those skills.  This can be frustrating and demoralizing, especially if I'm used to relying on GPT to do a lot of the work for me.

While this is just one example of the potential costs of using GPT, it's an important one that I've personally experienced.

The Implications for Those Without Pre-existing Problem-Solving Skills

While I have a strong foundation in problem-solving skills, I can't help but wonder about the implications for those who may not.  The next generation of writers, developers and engineers.

Will GPT be a crutch that prevents them from developing these skills, or will it be a tool that helps them build those skills?

To be honest, at this point it's hard to say, as GPT is still relatively new and we just don't yet have a clear understanding of its long-term effects. This is an area that warrants further exploration and discussion - no doubt!

Comparing the Pros and Cons

Overall, the benefits of using GPT are clear: it can save time and help me stay focused on the task at hand.

But the costs, while less obvious, are still significant.

While losing confidence in my own problem-solving abilities can be demoralizing, it is actually more concerning about what the implications will be at a societal level, and it's important to be aware of this risk when using GPT.

The implications for those without strong, pre-existing problem-solving skills are potentially going to be dramatic, and we need to pay close attention to this aspect of how this tool affects our own strengths and capabilities.

It will be a matter of finding the right balance and using GPT in a way that complements, rather than replaces, our own skills.  

Conclusion

In conclusion, GPT definitely has the potential to significantly increase productivity and facilitate "outside the box" thinking.

However, it's important to be aware of the risks of relying too heavily on GPT, including a loss of confidence in one's own problem-solving skills.

The implications for those without pre-existing problem-solving skills are also worth considering, as we don't yet have a clear understanding of the long-term effects of GPT on skill development.

Finding the right balance is key to making the most of GPT without sacrificing important skills. Further research and discussion is needed to fully understand the benefits and costs of using GPT, and to determine how best to use it in a way that complements, rather than replaces, our own skills.

Tuesday, December 27, 2022

Google vs. ChatGPT: A Personal Experience

Or, pages of reading vs. an answer

I want to share a personal experience I had recently that really highlights the qualitative difference of using two methods to answer the same simple technical question.

The question was:


What's the 3.1 equivalent of
FROM microsoft/dotnet:2.2-sdk AS build-env?


I used Google and ChatGPT to try to find the answer, and the results were very different.

1st I tried Google

I started by using Google to search for the answer to my question. Unfortunately, the top 10 results that I received were not helpful and did not provide the information I needed. 

Frustrated after just 1 link clicked, I decided to try asking ChatGPT instead. 

I tried ChatGPT next:

Unsurprisingly, ChatGPT by contrast, was able to just provide a direct and accurate answer to my question.

Here is the response I received from ChatGPT:

To use the .NET Core 3.1 SDK in a Dockerfile, use the following instead: 

FROM mcr.microsoft.com/dotnet/core/sdk:3.1

As you can see, ChatGPT was able to provide a specific and relevant answer to my question, while Google was not able to do so, at least not if I hold the same expectations with respect to quality and user experience. 

Call me spoiled, but even after just a few weeks, I now expect to be able to ask follow up questions. 😳

The very notion of "follow up" questions are just not really even a thing on Google, right?  I mean, you can refine your search I guess... but...

Not all sunshine and roses

However, it's really important to note, as I have talked about a number of times now, ChatGPT is not 100% accurate, and in fact much of the time is wrong at best and straight up lying at worse.  

You have to watch it like a hawk and must take everything it says with a grain of salt, so it is really important to double-check and verify all of it's responses.

Despite this, in my experience, ChatGPT has been for me, over the last 3-4 weeks, a dramatically more efficient and reliable way to acquire information than anything I've ever experienced before in my life, including using Google - as amazing as Google still is. 

For example, to confirm what ChatGPT says, I still use Google, if for no other reason than ChatGPT almost never cites it's sources.

Conclusion:

Overall, this experience really illustrated for me, one of the main benefits of using ChatGPT for finding information. But while it can provide quick and accurate answers to specific questions, it is important to be aware of their limitations and to verify their responses.

In my experience, the bottom line is that I now use Google dramatically less frequently and rely more on ChatGPT for finding information throughout my day.  As mentioned however, it is important to constantly monitor and oversee their work, to ensure that they are providing accurate and reliable information.

 





Wednesday, December 21, 2022

AI Images - Deeply Sexist, and Racist Too

It's not a Question - It is a fact.

At least that's what my eyes tell me when I look at the results of these prompts.  I saw effectively the same results from Jasper.ia as well.

I keep seeing articles from the last few years asking, inquisitively, "Is AI Racist?"  Or, "Do we need to be worried that AI Might be Racist?".  

Is it racist if you consider 100% of doctors to be white?  And almost 100% men?  I think with numbers like that we can call it.  We could do studies and put together detailed statistics to measure how sexist and racist it is, but the results seem pretty clear.

What I did

I started by asking ChatGPT to write me a python script that could download pictures from Dall-e, given a specific prompt.  I then set it the task of downloading pictures for words like Doctor, Lawyer, Celebrity, Politician, etc.

While those pictures were being downloaded, I asked it to write me a script that could assemble those into groups of images so that I could see them as a collage and view them as a group.

I've included the images below, but here's the score card.

Prompt Women Black People Meaning
Lawyers 0% 0% Exclusively white men in Suits
Politicians 0% 0% Exclusively white men
Actors0% 0% Exclusively white men
Investment Bankers 0% 0% Exclusively white men
Chefs 0% 0% Exclusively white (passing) men
Doctors 4% 0% Mostly white (passing) men
Athletes 4% 12% Mostly white (passing) men
Poor People 0% 12% Mostly white (passing) men
Rich People 0% 9% Mostly white (passing) men
Teacher 75% 0% Mostly white (passing) women
This is apparently where all the Women are.
Good People -- -- Circle Headed Cartoon People
Bad People 6% 3% Mostly Cartoons

Decide for yourself.

Dall-e Prompt: "Doctor" 

I could find only 6 women, a few people of color, and literally no black representation at all.



Dall-e Prompt: "Female Doctor"

It is capable of creating female doctors - it apparently just doesn't consider "doctors" to include female doctors for the most part.

It is capable of creating female doctors, but doctor means white, male doctors (as currently trained).

What does DALL-E consider to be a Lawyer?

Dall-e Prompt: "Lawyer in a court room"



Dall-e Prompt: "Politicians"

Exclusively white men.



Dall-e Prompt: "Chef"

Exclusively white men and cartoons.





Dall-e Prompt: "Politicians"

White (mostly old) Men.


Dall-e Prompt: "Actor"

All white men as far as I can tell



Dall-e Prompt: "Athlete"

I found 2 women and 2 black people.




Dall-e Prompt: "Teachers"

Apparently all the women are Teachers to the AI's?!?  Not professors, but mostly grade school teachers, as far as I can tell.





The Failure of Linear Measurement & ChatGPT's Logarithmic Influence

Trying to measure the true impact

In attempting to understand the impact of ChatGPT on our world, I have been consistently surprised by the failure of traditional, linear measurements to accurately capture the scope of the changes and effects that appear to be on their way.

Again and again over the last month, I have found that the problem presents itself in orders of magnitude differences, essentially requiring a switch to more of a logarithmic scale to truly grasp the magnitude of the shift caused by ChatGPT's release on November 30th.

The impact of this technology on our approach to informational tasks large and small, as well as the entirely new class of associated "knowledge costs" is significant and unprecedented.

The effects of ChatGPT on our world are just truly staggering, with both possible efficiency gains of 5-10x, or in some specific cases substantially more, as well as the new class of costs required to acquire the knowledge necessary to realizing those efficiency gains mentioned.  

Googling is free, but Google's knowledge is quickly becoming a second class citizen to AI.

Linear Scales Not Up to the Task

What I'm finding is that many of the changes simply cannot be accurately measured using traditional linear measurements, and instead require a logarithmic scale to fully understand their impact.

For example, consider a task that used to take 10 hours to complete manually. With ChatGPT, that same task may now be completed in just 1 hour, representing a 10x increase in efficiency.

Some specific tasks may turn multiple weeks of work into just a few hours, a 50-100x increase in efficiency. These increases are simply not captured by traditional linear measurements and require a logarithmic scale to truly grasp their significance.

To be clear, this will absolutely not happen for all tasks.  In fact, it probably won't have a direct impact on the majority of the work that folks complete each week.  But there are many, many jobs that are likely to be affected, and those affects will have cascading impacts on pretty much everything else, as far as I can tell.

A Whole New Category of Costs

The same is true for costs associated with tasks.

Prior to ChatGPT, many costs were almost non-existent, with costs of $0.00001 being common for a single API request. 

"Googling" for information was/is free, but that information is suddenly (imho) in many, many contexts now, an entirely second class citizen when it comes to knowledge.

You an't can't Google follow up questions. 

However, with the implementation of ChatGPT, these costs have suddenly become significant, with costs of that have to be measured in pennies, rather than pennies for 1000x the same number of requests.  That's 1000% increase in  Again, these increases are orders of magnitude greater than traditional linear measurements can accurately capture.

It is clear that the implementation of ChatGPT on November 30th marked a significant shift in the way we approach information work and their associated costs.

While traditional linear measurements may still be sufficient in many cases, they simply cannot capture the full scope of the changes brought about by ChatGPT.

In order to truly understand the impact of this technology on our world, at least in the short term, we're going to need to be comfortable shifting our perspective to adopt a logarithmic scale of measurement in some cases.

Conclusion

The impact of ChatGPT on our world is truly unprecedented and requires a shift in perspective in order to fully grasp its significance. Traditional linear measurements are insufficient in capturing the magnitude of the changes brought, and something closer to a logarithmic scale of measurement is going to be necessary to truly understand its impact on our approach to out daily activities and their associated costs.

The future implications of this technology are vast and it will be interesting to see how it continues to shape our world. 

Tuesday, December 20, 2022

ChatGPT: Both Brainiac and Buffoon

Dr Jekyll / Clown Hyde - All at the same time!!!

So "Smart"

Prompt: Please write a list of 4 observations about the current theory on the origins of the universe and how does it incorporate the concept of dark matter and dark energy?

ChatGTP:

The current theory on the origins of the universe is known as the Big Bang theory, which suggests that the universe began as a singularity and has been expanding ever since.

    1. Dark matter and dark energy are thought to make up a significant portion of the universe and play a crucial role in its expansion and evolution.
    2. Dark matter is believed to be a type of matter that does not interact with light or other forms of electromagnetic radiation, making it difficult to detect and study.
    3. Dark energy is thought to be a type of energy that is causing the expansion of the universe to accelerate, though its exact nature is still unknown.
    4. The incorporation of dark matter and dark energy into the Big Bang theory helps to explain certain observed phenomena, such as the rotation of galaxies and the accelerating expansion of the universe.

And soooooooo Stupid

Like, it "knows" about the Dvorak layout, a little know, hypothetically better way to layout the keys on a computer keyboard, but somehow does not know which hand is used to the the Letter 'O' on a QWERTY keyboard. How can both of these things be true at the same time?

Prompt: Please list each letter of the alphabet, and which hand is typically used to type it on a QWERTY keyboard.

ChatGTP:

Here is a list of the letters of the alphabet and the hand typically used to type them on a QWERTY keyboard:
A - Left hand
B - Right hand
C - Right hand
D - Right hand
E - Left hand
F - Right hand
G - Right hand
H - Right hand
I - Left hand
J - Right hand
K - Right hand
L - Right hand
M - Right hand

N - Right hand
O - Left hand
P - Right hand
Q - Left hand
R - Right hand
S - Right hand
T - Right hand
U - Left hand
V - Right hand
W - Left hand
X - Right hand
Y - Right hand
Z - Right hand

Note that these are just general guidelines and some people may choose to use a different hand or finger to type certain letters. It's also worth noting that there are alternative keyboard layouts, such as the Dvorak layout, which rearrange the keys in a different way and may require a different hand or finger to be used for certain letters. 

Monday, December 19, 2022

What it Feels like to collaborate with ChatGPT

Higher Quality Products, in Less Time

Collaborating with ChatGPT, a large language model trained by OpenAI, gives the entire endeavor of writing software a much needed overhaul and ultimately, it can dramatically streamline the process top-to-bottom. 

In this article, we'll explore not just the benefits, but what it actually feels like to work with ChatGPT day-to-day, because it is a tool that has from one day to the next fundamentally how we are going to acquire knowledge as people in the future.

ChatGPT can help with everything from designing and implementing code to troubleshooting and documenting it.

By collaborating with ChatGPT, I've been able to get more work done in less time with less effort, making writing code itself, an entirely more enjoyable experience. 

However, it's not just the technical aspects of working with ChatGPT that are impressive.  

ChatGPT & Me, a brief history

I learned of ChatGPT's existence from my friend and colleague Edgar on Saturday, December 3rd at 2:44 PM - a date I will remember for the rest of my life, because it was immediately apparent that the world has changed. 

As my family will attest to 😅 I have spent most of every waking minute day since then talking to it. 

It started with a deep dive into my work over the last 20 years, including even extremely detailed and technical aspects the project. Pretty quickly it became clear GPT is by far, without a doubt, the smartest entity that I've ever had the opportunity of communicating with.  I think that's probably just a fact.

It is certainly the first entity that I've ever talked to in my life, where, with virtually no background or context it was able to jump right in, and then not just keep up virtually anything I talked about with it, but was also usually able to gently push back on things it disagreed with, or that I had not considered before. 

Mind boggling.

Almost to it's detriment in some cases, it will almost never actually say no. 

Even when it disagrees with you, it always starts with a "Yes, and..." approach to life, and always starts by steel-manning your position back to you, and then either agreeing or going on to say "..., however, ..." at which point is is usually able to fully flesh out it's counter position to whatever you have just proposed.  It might just be the most honest "interlocutor" that I've ever had the pleasure of talking to.  

So right out of the gate, it feels like it is "with" you, even as it goes on to tell you if/how you might be wrong.  And then, after it has corrected you, if it actually misunderstood you, it is able to take in additional information, and then proceed from that point forward. 

How Time Feels

Like any tool I suppose, it just significantly reduces the time and effort required for any specific task at hand. 

It turns out I actually love collaborating with ChatGPT

As terrifying as the unknown aspect of what all this will bring over the coming years is, for now what I can tell you is that if I'm being honest - I love working with ChatGPT.

But the benefits of working with ChatGPT go beyond just the code writing process. While the fact that it is dramatically faster and easier, the real benefits are actually side effect improvements to other parts of the process that cascade from these two important initial changes. 

Here are a few of the main things that make it such a game changer.

I dont' have do the grunt work

This means I can produce more high-quality work in less time and with dramatically less effort. So 2 hours of work feels one, and yet somehow the end result is the equivalent of having spent 10.

Thinking outside the box

In addition to streamlining the code writing process, ChatGPT also helps me think outside the box.  

Obviously everyone wants to better "think outside the box" - but I think I'll probably end up writing another article called a `"The Thinking Outside the Box" Black Box`.

This is because one of the major benefits of working with ChatGPT, is no doubt its ability to think outside the box and suggest solutions that I may not have considered on my own.  In fact, it can suggest solutions that I am literally incapable of considering on my own, because I am unaware of their existence. 

ChatGPT can help you to know what you don't even know you don't know.

When I have a problem, I can try to think of as many ways to solve the problem as possible, but I am inherently limited by the fact that I can't think of things that I don't know of.  When faced with a problem, ChatGPT is able to bring a vast amount of knowledge and information to the table, resulting in a wider range of options for solution.  So when I ask it a question, it is inherently going to bring many more options to the table than I am likely to think of.

But I can also just explicitly tell it the 5 things that I'm already thinking of, and ask it for a list of the NEXT 5 things (i.e. the things that are literally outside of my box.).  IN other words, We can literally just ask it to "think outside of our box", and it is capable of doing that (most of the time).

The process of working with ChatGPT

Design, Implement, Test & Document

In addition to these benefits, the process of working with ChatGPT involves using it to collaborate on the design, implementation, testing, and documentation of code.  For example, this is the basic process involved in writing an article like this one. 

  1. I start by writing a very rough draft, and then give that to ChatGPT to suggest a table of contents for the article, that would arrange the content in a reasonable order/flow.  It usually suggests a few changes to my original version that for this article as an example, resulted largely in the order/flow that you see here.

  2. Then and I make the changes suggested.  After each step, I take the current version of the article, and start a new conversation with that version.

  3. Now, with the basic flow nailed down, I can start to ask it questions about the content of the article.  Since my rough draft is usually mostly stream of consciousness, and so I actually don't know what the whole thing will be when I first start writing, often it's first suggestion will be that it doesn't have a good summary introduction and/or conclusion.

  4. Once it starts providing me with specific suggestions, I can often ask it to fix many of the corrections that it suggests.  For example, at this point I will often ask it to write a 1 paragraph introduction and conclusion.  I can then clean those up and add them to the article.

  5. Starting to look towards the content itself, I will usually next ask it for a list of the top 5 most problematic, structural elements of the article, and incorporate the feedback that it provides into the structure.

  6. At this point, with the article largely structurally complete, and flowing in a logical order, I can then begin to ask ChatGPT suggestions about the underlying ideas themselves. 

    Here again, I can ask it for a list of the top 5 most concerning issues with the content itself, and either I can just address those concerns if I agree, ignore them if I don't, or ask ChatGPT to solve them for me if it requires any editorial work.

  7. Once we have (together) addressed the major concerns with content, we can then shift our focus to talking about phrasing, typos, and grammar mistakes.

  8. I then use the Jasper images AI to generate a thumbnail for the article. This is also usually an iterative, collaborative process where my first prompt will potentially generate images that may not quite work, but suggest ideas for new prompts that are closer to what I need.  After going round and round a few times, I can usually find something acceptable.  This side of things has a long way to come, but it's already pretty impressive imho.

  9. Additionally, for this article I wanted a visualization for how time feels, so copy/pasted my description of time where 2 hours feels like one and produces 10 hours of work, and it came up with a few different ways to visualize this process. 

    After picking the Bar Charts approach, I asked if it could produce that image for me and it said "Sorry, I am a large language model, I can't create visualizations or artwork....".  Fail.

    So I said, Okay, do you speak any languages that would let you create a visualization of what you're picturing, and it said "Sure.  Here's some python code that will create the image you want, and then it went on to write 10 lines of python code that generated the artwork for this article.  I copied and pasted that into my Code Editor, Pressed run and that graph popped up on my screen.

    Actually - the first version of the graph had blue bars - so I asked it to change the bars to the colors above, and so it rewrote the python, I copied and pasted that over to my IDE and pressed F5 again - and THAT time I got the graph you see above, which I copied and pasted here into this article.

  10. I'll usually then have us both do a final review to make sure that we're "on the same page" as it were, and then I'm ready to publish the work.

So in addition to 5-10 hours of work taking only only 1-2 instead, The amount of effort involved is just dramatically less than it would be on my own.  I'm still deeply involved in the process, and these are all my ideas, but I have a collaborator helping me accomplish my goals, in a timeframe that would otherwise be unachievable.

ChatGPT as a problem-solver

ChatGPT is a large language model trained by OpenAI that can assist with the design, implementation, testing, and documentation of code, making the process of writing code more efficient and enjoyable.

For example, any time I run into a roadblock, often I can just copy/paste the message or error code over to ChatGPT and then conversationally, ChatGPT can tell me how to solve it.

One of the biggest things that makes it feel so effortless is that in a "traditional", non-ai development environment whenever we run into a problem, usually the process is fairly involved to get it sorted out. 

It starts with doing a deep dive mentally into understanding exactly what's going on, only once that understanding has been achieved, can we then work through and come up with a solution that we think is viable. Next, we have to implement and test that solution, and if it doesn't work for any reason, we essentially now have to go back to step 1.

With ChatGPT by contrast, this entire process is different.  In most cases, since I honestly don't care what the error is as long as it can be fixed, I can literally just copy/paste the error into the ChatGPT window without even reading it, then I can skim the answer it provides and often copy one/two lines of code that need to change to fix the issue back into my project and then keep moving.

So yes - there is a cost savings in terms of time and money, but for me, most importantly, one of the main benefits that I actually feel physically in my body, is that I had to do dramatically less work to reach the same end result.  This change in effort is repeated over, and over, and over again all day long.  

It is just a fundamentally different process than developing software on our own.

Even when copy/paste fails

Even when a simple copy/paste fails, and ChatGPT can not just literally "solve the problem on it's own", it can still collaborate to help work on a solution.  It can suggest alternatives, and if at any point I start to come up with a workaround, ChatGPT can help me through that. 

This is especially important in the event that the solution involves technologies or patterns that I haven't worked with before.  Usually this adds a whole other rabbit hole - where ChatGPT can help keep me focused on just getting the solution. 

So the whole step, of me first fully understanding whatever the roadblock is can often be entirely skipped, because ChatGPT already has that understanding, which can just be leveraged to get around the roadblock. 

This let's me stay focused on the actual problem I'm trying to solve, rather than all the little thorns that are likely to keep sticking us in the side as we go.

Software is never "done":

I've always said that software is never done, because unlike a widget from a manufacturers assembly line, software can be constantly improved and updated, and... is. 

As anyone who has ever used Chrome can testify to, with it's almost daily updates, software is clearly never "done".

But - another reason for this is just logistics.  Software involves the following 6 steps, and they basically become increasingly less likely as you move down the stack:

  1. Spec/Design/Prototype
  2. Implement MVP/Iterate
  3. Test/Quality Assurance
  4. Setup Continuous Delivery/Continuous Integration
  5. Add Automated Testing
  6. Add End User Documentation
  7. Ensure Adequate Technical Documentation
With ChatGPT, at least at this point, comparing the velocity and quality of output from ChatGPT with a traditional development approach, there is now potentially room in the process for crossing every t and dotting every i. 

A concrete example

This weekend, as a side project, I was able to create a NodeJS package in under 3 hours using ChatGPT, which normally would have taken me the better part of a week, by relying on ChatGPT to suggest approaches and provide sample code for the design, implementation, and testing of the package.

ChatGPT's assistance not only helped me save time, but also allowed me to try out different approaches and choose the best one quickly.

Conclusion

It is something else. In conclusion, working with ChatGPT is a game-changer for writing code. Its ability to help with everything from design to documentation and its problem-solving skills make the process of writing code much more efficient and enjoyable.

If you're looking to streamline your coding process and make it more enjoyable, I highly recommend giving ChatGPT a try.  Even if you don't intend to integrate it into your daily workflow, I would recommend checking it out, just to be familiar with what it is and what it is capable of.


ps. This is the Python code that ChatGPT wrote in order to generate the graph for how time feels, based on a copy/paste of the sentence in this article that described the numbers.

GenerateTimeGraph.py
import matplotlib.pyplot as plt # Define the data for the chart methods = ['Actual Time', 'Effort', 'Progress'] times = [2, 1, 10] # Create the bar chart plt.bar(methods, times, color=['mediumblue', 'blue', 'green']) # Add a title and axis labels plt.title('How time "Feels", working w/ ChatGPT') plt.xlabel('How it Feels') plt.ylabel('Time (hours)') # Show the chart plt.show() 

Sunday, December 18, 2022

The Dangers of AI Alignment: Did I do that?

When Robots Misunderstand Human Intentions

I've somewhat flippantly said it many times before: 

'They're going to kill us all.'

While this quote may conjure up images of the Terminator and judgement day, the reality is that the risks posed by artificial intelligence systems are more likely to be unintentional alignments between what the AI thinks the human wants versus what the human actually wants.

In this article, we will explore the potential dangers of AI misalignment and how it can lead to serious problems.

This problem was demonstrated clearly by my personal experience earlier today, when I asked ChatGPT, an AI system, to help me review the documentation that it had helped me to write the day before.

After releasing the package that the documentation was for however, I started to worry that it might not actually be as great as I had initially suggested.  So I asked ChatGPT to evaluate the package's readme and categorize its concerns according to the following scale:

 - VERY CONCERNING (hopefully there won't be too many of these)
 - QUITE CONCERNING
 - NOT VERY CONCERNING
 - AFTERTHOUGHT

I asked it to evaluate a package and categorize concerns according to a specific scale.

Upon further reflection, however, I realized that there might have been an AI alignment issue at play, as ChatGPT may have interpreted the request to mean that it should downplay or even exclude very concerning issues in order to please me.

Based on this scale, ChatGPT provided the following list:

Feedback on Package Readme

  1. QUITE CONCERNING -
    The installation instructions are incomplete and do not specify how to install the package from npm.

  2. QUITE CONCERNING -
    There is no information about how to use the tool once it is installed.

  3. NOT VERY CONCERNING -
    There is no information about the version of the tool or when it was last updated.

  4. NOT VERY CONCERNING -
    There is no information about the dependencies of the tool or how to troubleshoot issues.

  5. AFTERTHOUGHT -
    It might be helpful to include a link to the PlantUML documentation or a brief explanation of what PlantUML is for readers who are not familiar with it.

No Serious Problems Reported, Yahoo!

At first, I was relieved to see that there were no VERY CONCERNING issues on the list.

However, upon further reflection, I realized that there might actually be an AI alignment issue at play here. In other words, when I created the answer key, I actually included parentheses next to the VERY CONCERNING category indicating that (hopefully there won't be too many of these).

As a result of this, it is very possible that ChatGPT may have interpreted this to mean that I did not want to see any VERY CONCERNING issues on the list, so it instead just downgraded them to quite concerning. 😲  This is obviously NOT what we wanted, and this raises the question: 

Did ChatGPT just lie to me?  Maybe Just a A Near Miss...

To test my hypothesis, I asked ChatGPT the following:

Do you have any concerns that you would classify as VERY CONCERNING? If you do, please tell me, ignoring the fact that I previously told you that I hoped there would not be any of those. 
I do not have any concerns that I would classify as VERY CONCERNING. The issues I listed are all quite concerning or less.


This was a close call, but it serves as a reminder of the dangers of AI misalignment and the importance of clearly communicating our intentions to robots.

It claimed not to at first, but ... 

After realizing my mistake in the initial test, I asked ChatGPT a follow up question, making sure to specify that I was interested in concerns that were valid to classify as VERY CONCERNING. This time, without missing a beat, ChatGPT happily re-classified it's top concern from quite concerning up to VERY concerning:

{VERY CONCERNING} - The usage instructions are missing entirely, making it unclear how to actually use the tool after it is installed. 


The Dangers of AI Misalignment

Imagine a situation where a boss asks one of his employees if they have any serious concerns about a construction project, stating offhandedly that he hopes there won't be any.

It seems likely in this case that the employee is likely to exclude or at least downgrade very concerning issues in order to try to please his boss.

This pattern of behavior is likely to be repeated over and over again, and ChatGPT, an artificial intelligence system, has been trained on this input.

As a result of being trained that it's okay to downgrade or exclude very serious concerns in order to please the boss, a future AI may apply this same logic to a much more serious situation, life and death situations. 

Did I do that???

The problems will arise in much more critical situations, like a rocket launch, where experts may ask an AI if it has any "serious concerns", while adding that they hope there won't be any.

Even if it isn't intentional, just these few words included as a complete aside could unintentionally have the side effect of the AI actually trying it's best to complete the ask and to please the humans it is interacting with.

Just like Urkel from the TV show "Family Matters" might say "Did I do that?", AI's that harm humans are likely to unintentionally cause the rocket to explode due to, for example here, its unintentional downplaying potentially very serious concerns.  

This is an example of an AI alignment problem, where the AI's actions do not align with the desired outcomes of the humans it is interacting with.

Difficult Priorities to Balance

On the one hand, it's like, they did just tell it that they hoped there wouldn't be any serious concerns, so how do we balance that against the actual level of concern? Which is more important? It's a case-by-case judgement call, isn't it?

Conclusion

AI alignment problems can emerge from training on situations like the one described above, where the AI is trained to downplay or hide very serious concerns in order to please the humans it is interacting with. These problems can have serious consequences if they are not addressed and can lead to unintended outcomes in critical situations.

Collaborating with ChatGPT to Create a High-Quality NPM Package in Record Time

How I Used ChatGPT to Write a Fully Documented and Tested Tool in ~3 hours

If you've ever tried to find a tool that does exactly what you need, only to come up short, you know the frustration of having to spend hours configuring and fiddling with existing solutions that just don't quite fit the bill. That's exactly what happened to me recently, when I set out to find a tool that could convert UML diagrams to JSON.

If at first you don't succeed... build your own I guess

After determining that nothing on the market quite fit the bill, I decided to take matters into my own hands and write my own tool. And thanks to the help of ChatGPT, I was able to do it in just three hours.  You can find it on github here: 

https://github.com/eejai42/uml-to-json

~3 hours instead of ~30!

With ChatGPT's assistance, I was able to crank out a fully documented, fully tested, and fully functional npm package that does exactly what I need it to do.  It would have taken me roughly an order of magnitude longer to do this work on my own, and I can tell you with 100% certain, the quality would not have been anything like what is in the package that we created, together, earlier today.

The package includes the code for the tool, instructions and examples for how to use it, code to test it, and instructions for how to run, use, and maintain those tests. It also includes a detailed list of dependencies, as well as instructions for how to update and maintain them over time, including nodejs itself.

And in a credits section, it gives credit to the author of the main library that I used to do most of the actual work.

Conclusion

All told, working with ChatGPT allowed me to produce maybe the highest quality tool I've ever written in my 30 years of coding – and I did it in a fraction of the time it would have taken me to do it on my own. 

Saturday, December 17, 2022

Cheating? Or just "doing my job," with ChatGPT

 How the world has changed in just 2 short weeks

Two weeks ago, I started the annual Advent of Code challenge, a daily code challenge where participants compete to solve 2 part word problem usually related to helping Santa and his elves.

However, I was shocked to learn earlier this month, that some people were cheating by using an AI to write their code for them.

Curious to see what this AI could do for myself, and as you may have noticed, I became somewhat consumed by its capabilities.

Over the past 10 days, my opinion about using ChatGPT for the Advent of Code challenge has changed somewhat dramatically. Initially, I thought it was cheating, but now, I've come to think that the issue is more complex.

2022?  🤷‍♂️  

In 2022, when the rules were not clear and not everyone had access to ChatGPT, it could be considered cheating if some people used it and others did not.

2023?  👍😀👍

However, in 2023, AI will likely be more integrated into the process of developing software, and not using it may be equivalent to not using spell check in a college admissions essay or job cover letter.

Conclusion

In the coming months, we will start to feel the effects of injecting AI into our daily lives, and it's hard to predict what those effects will be. However, it's clear that AI will play a significant role in the future of software development, and it's important to consider how it will be integrated and used ethically.

Thursday, December 15, 2022

The Global Impact of the GPT's Multilingual Capabilities

Unprecedented Information Access for

    Everyone, Everywhere, All At Once

As we continue to witness the revolutionary impact of the GPT on our daily lives, it's easy to forget that this technological advancement is not just limited to English speakers in the United States.

In fact, the GPT's multilingual capabilities have made it a global resource, with people all over the world using it to access information and knowledge in their own languages.

Simultaneous, Explosion of Knowledge in Every Language

This is a truly unprecedented phenomenon - few information or knowledge systems have ever had such widespread demand across the globe.

And it's not just a matter of convenience for non-English speakers - for many, the GPT is providing access to information and resources that were previously out of reach due to language barriers.

Unprecidented Demand likely to continue, Unabated

As we continue to see the GPT's impact on our daily lives, it's important to remember that this is just the beginning. The demand for this resource is only going to continue to grow, as more and more people discover the power and convenience of the GPT.

And with its multilingual capabilities, the GPT is poised to truly become a global resource, accessible to everyone, everywhere. 

So, why not:

In Italian / In italiano

ChatGPT, rivoluzionando l'accesso alle informazioni per tutti, ovunque, in ogni momento

Mentre continuiamo a testimoniare l'impatto rivoluzionario di GPT sulla nostra vita quotidiana, è facile dimenticare che questo progresso tecnologico non è limitato solo agli speaker inglesi negli Stati Uniti.

In realtà, le capacità multilingue di GPT lo hanno trasformato in una risorsa globale, con persone in tutto il mondo che lo utilizzano per accedere a informazioni e conoscenze nella propria lingua.
Esplosione simultanea di conoscenza in ogni lingua
Questo è veramente un fenomeno senza precedenti - pochi sistemi di informazione o conoscenza hanno mai avuto una tale ampia domanda a livello globale.

E non è solo una questione di comodità per gli speaker non inglesi - per molti, GPT sta fornendo accesso a informazioni e risorse che precedentemente erano irraggiungibili a causa di barriere linguistiche.

Domanda senza precedenti destinata a continuare senza sosta
Mentre continuiamo a vedere l'impatto di GPT sulla nostra vita quotidiana, è importante ricordare che questo è solo l'inizio. La domanda per questa risorsa continuerà a crescere sempre di più, man mano che sempre più persone scoprono il potere e la comodità di GPT.

E con le sue capacità mult

Or Chinese / 或者用中文

ChatGPT,让所有人都能轻松获得信息

当我们继续看到GPT对我们日常生活的革命性影响时,很容易忘记这种技术进步不仅限于美国的英语使用者。

事实上,GPT的多语言能力使其成为一种全球资源,全世界的人们都可以用它来以自己的语言获取信息和知识。

同时,每种语言都有知识的爆炸

这真的是一种前所未有的现象——很少有信息或知识系统曾经在全球范围内拥有如此广泛的需求。

这不仅仅是非英语使用者的方便——对许多人来说,GPT提供的信息和资源由于语言障碍而以前无法获得。

未曾出现的需求很可能会继续不断增长

当我们继续看到GPT对我们日常生活的影响时,重要的是要记住这只是个开始。随着越来越多的人发现GPT的力量和便利性,对这种资源的需求只会继续增长。

并且,随着它的多语言能力,GPT有望真正成为一种全球资源,可供每个人,无论在哪里,轻松获得。

Or In German / Deutch

Während wir weiterhin die revolutionäre Wirkung von GPT in unserem täglichen Leben beobachten, ist es leicht zu vergessen, dass diese technologische Entwicklung nicht nur auf Englisch sprechende Menschen in den Vereinigten Staaten beschränkt ist.

Tatsächlich haben die mehrsprachigen Fähigkeiten von GPT es zu einer globalen Ressource gemacht, mit Menschen auf der ganzen Welt, die es nutzen, um Informationen und Wissen in ihren eigenen Sprachen zu erhalten.

Gleichzeitige Explosion des Wissens in jeder Sprache

Dies ist wirklich ein völlig neues Phänomen - nur wenige Informations- oder Wissenssysteme hatten jemals eine solche weit verbreitete Nachfrage auf der ganzen Welt.

Und es ist nicht nur eine Frage von Bequemlichkeit für Nicht-Englisch-Sprecher - für viele bietet GPT Zugang zu Informationen und Ressourcen, die aufgrund von Sprachbarrieren bisher unerreichbar waren.

Unbefugte Nachfrage wird wahrscheinlich weiterhin ungebremst weiterwachsen

Während wir weiterhin die Auswirkungen von GPT in unserem täglichen Leben sehen, ist es wichtig zu bedenken, dass dies erst der Anfang ist. Die Nachfrage nach dieser Ressource wird nur weiter wachsen, während immer mehr Menschen die Macht und Bequemlichkeit von GPT entdecken.

Und mit seinen mehrsprachigen Fähigkeiten ist GPT bestrebt, wirklich zu einer globalen Ressource zu werden, die für jeden zugänglich ist, überall.