Created our own GPT and here is what we learned

We created our own GPT (a delivery readiness evaluation) and deployed it in OpenAI's GPT Store last week. The experience was unique enough that we wanted to share some of the experience as well as what surprised us and what disappointed us, so here are the highs, lows and buffalos.

Background on the app we created:

We have an evaluation framework we call CLeAR which has 18 questions (in the short version) segmented across the CLeAR dimensions (Culture, Leadership, Alignment, Responsiveness, Resources and Readiness) which we use to determine strengths and areas of opportunity for an organization to deliver change (products or projects) effectively. We have a free version of CLeAR on our website and will send anyone who takes it a RADAR chart of where they are strong or need improvement. We invested time in CLeAR last year to make it clear (pun intended), to simplify it and wrote some code to automate some of the steps. My main point is, we invested time in CLeAR and had some great inputs for our GPT even though it was never an expectation we'd create one.

You do need a paid version of chatGPT in order to build GPTs or access GPTs in the GPT store. Either Plus, Team or Enterprise. I've looked for stats on how many of the 100's of millions of users are paid or not paid but it's not public info so I am not sure how large that population is.

Highs:

• Creating your own GPT is super easy! We created our GPT and launched it within a couple of hours. No coding required. Just configure and launch but it is helpful if you have some unique or proprietary content to drive the user experience.

• Launch something super fast! What would normally have taken 2-4 weeks took me 2 hours! It is truly impressive that I can take content from a process we created and documented, upload it and essentially give it logic (in natural language) on how to interact with someone. This would have been a decent sized effort in the past to take our unstructured content and put it into a structured DB and create the logic for how it should behave and code basic screens and prompts.

• It has intuition. OK, maybe it isn’t intuition, but I configured our GTP to ask for answers in the form of a number ranging from 0 – 4. I tried some boundary testing and entered 5, even though it instructed me to use 0-4. My GPT politely thanked me and asked me to provide an answer between 0-4. From someone who wrote code in the past, I found this awesome.
• A new way to engage with content. A custom GPT could be a great way to expose others to your content. Do you have a methodology, process, framework, playbook, etc that you share with your team or customers? Our GPT, will tell you all about our evaluation framework and (based upon my testing) it is very accurate. However, it will not give away your documents. I’ve asked for content I uploaded but rather than giving me the actual content it will only speak to it and direct to visit www.neuzeitgroup.com for more information.

Lows:

• Controlling the user experience: You don't get to control the user experience which is a non starter for some applications. For instance, I instructed the GPT to ask the questions from our eval one at a time. The first time I tested it, it asked them in batches of 3 based upon our categories. I didn’t expect this. It worked, but it was not what I expected. Other tests it did ask one question at a time. I also instructed my GPT to tell the user how many questions they had remaining. It did this 1 out of 3 times. Good or bad really depends upon your use case. This could be a non-starter for applications that require a very specific and consistent experience.
• Developer experience is lacking: I once developed apps for the iPhone and Apple spent a great deal of time before opening their app store in considering the developer experience. Everything from naming, guidance, testing, documentation. Developing a solution in the GPT store and the Apple app store are very different, but I assume OpenAI will be enriching this experience. A simple example is I hit our GPT4 usage ceiling just in testing our GTP a few times which makes it hard to tweak and test.
• App store ecosystem and governance are completely absent. This store has only been available for a very short time but there’s already tons of garbage out there. I saw one person that created 500-ish GPT‘s on everything imaginable. It reminds me of the “I am Rich” app of 2008 that sold for $999 in the Apple app store and all it did was showed a jewel on your screen when you opened it. You also have no idea of who is using your app, their experience (reviews?) what region they are from, etc etc. I am sure OpenAI has plans for this too, but this could be a long journey until we have an ecosystem around creating your own GPT.
• Monetization. Many have commented on this but it’s up in the air on how Open AI will share the revenue they earn to the GPT Store community.

Buffalos:

• These are the unique experiences one did not expect. I mostly covered these but there is a cool factor to interacting with an app that has your content but represents it in its own unique way and the fact that it did things I would not expect of a machine but would of a human (if they were paying attention) were unexpected.

This is an exciting development, and it could dramatically change the way we interact with customers, put our content into marketplaces and build solutions but it will be a journey and take some time to get there.