I was curious to learn how ChatGPT achieves its “streaming” effect when it sends responses from the model back to the user. I’m referring to the cursor-driven effect that neatly “types” the model’s response in the chat interface:

Like any other practical hacker, I started by investigating the network requests tab in Chrome in order to better understand what’s happening when a user sends a message to the OpenAI server and ChatGPT responds. I expected to see some sort of simple HTTP requests or even a web-socket, but I was surprised to see neither.

As you can see in the image above, ChatGPT’s frontend communicates with the following APIs:

• /auth/session: Manages basic authentication.

• conversation/<unique conversation ID>: An endpoint that accepts HTTP GET requests and returns an existing conversation history.

• /conversations?offset=0&limit=20: A paginated HTTP GET endpoint to retrieve a set of past conversations and their metadata.

• /models: Returns the name and id of the models used with the current.

• /conversation: An endpoint that accepts HTTP Post requests from the user with the latest message, a conversation_id, the model being used, and a parent_message_id.

These APIs look pretty standard, but where is the response from the model? Given these network requests, I expected the request to /conversation to return an answer, but that was not the case. Notice how there is no “response” section in the selected request in the “screenshot above”.

This surprised me, so I assumed that ChatGPT is using web-sockets instead. However, when I looked for the web-socket-related network requests in Chrome, I found nothing. This started getting really confusing.

I started examining the response from each endpoint hoping to find something useful. I even considered that the server was returning responses in a base64 encoding, but I could find nothing that actually contained the response!

Server-sent events (SSE)

After about 15 more minutes of Googling and poking around, I realized that the ChatGPT server uses something called “server-sent events” (SSE) to send data back to the client. What tipped me off was the “EventStream” section in the /conversation endpoint. While the EventStream section was empty, it made me consider that something else was at play, and after a couple of searches, I learned that there was another way to send data to the client that did not include regular HTTP responses or web-sockets.

SSE is a very simple concept. It’s a one-way real-time connection from the server to the client that allows the server to push data to the client. This is similar to web sockets, but without being bidirectional (i.e. clients can’t respond). I had never heard of SSE before, but it looked like a very handy piece of web technology, so I decided to learn how to use it.

Setting up server-sent events is fairly easy. Since SSE is part of the HTTP standard, you don’t need a lot of tooling. You just need to add a listener on the front-end and a backend endpoint to send events from.

Here’s what the front-end code looks like for a basic implementation:

// JavaScript

// Create an EventSource linked to your page / endpoint.
var source = new EventSource("/sse");

// Add event listeners for "publish" and "error"
source.onmessage = (event) => {
    console.log(`event: ${event.data}`);
};

Setting up the backend code is equally simple. Here’s an example using Python and Flask:

# Python and Flask Pseudocode

@route("/sse")
def stream():
    def eventStream():
        while True:
            # Poll data from the database
            # and see if there's a new message
            if len(messages) > len(previous_messages):
                yield "data: 
                    {}\n\n".format(messages[len(messages)-1)])"
    
    return Response(eventStream(), mimetype="text/event-stream")

That’s it! Now you can use SSE too, like ChatGPT.

Many builders, investors, and users are now reconsidering their choices. When crypto came back with a vengeance and everyone was leaving mid-dinner to mint NFTs, the decentralized future felt certain. After a year plagued with scandals, hacks, and a worsening macro picture, things do not feel so certain anymore. Many builders, users, and investors in crypto feel demoralized and wonder if everything was just a fever dream.

While many people are reconsidering their excitement about crypto, I wanted to write about the reasons why I still feel optimistic about crypto as a founder. I will also share some advice we’re following during the next few quarters as we’re navigating the product-market fit maze with Marble, our own startup.

Reason #1: Dapps (on-chain “decentralized” apps) have gone viral for the first time

During the 2017 ICO boom and bust, many in crypto justified the insane valuations of different tokens and chains by comparing them to cloud providers that could run applications in a decentralized way. This quickly fell apart when people discovered that there were almost no apps out there yet. Furthermore, most use cases were still hypothetical (decentralized finance, NFTs, games, social, etc. were still all mostly in the idea realm).

To me, the most interesting takeaway from the 2020-2022 bubble is that we’ve seen dapps become viral for the very first time. By the end of 2021, wallets like MetaMask had over 30 million monthly active users and the number of daily active wallets connecting to dapps reached an all-time high of 2.7 million.

This started with DeFi summer, when apps like Uniswap, Compound, and Yearn proved that you could create successful on-chain applications that had real users and use cases. Dapp usage was turbo-charged in 2021, when NFTs became viral. Everyone was minting one, brands were getting involved, and marketplaces were printing cash hand over fist. NFTs showed that users were not afraid to use crypto applications. This sparked interest in a whole set of new use cases: consumer social applications, games, and even new concepts like DAOs were everywhere.

While usage numbers have fallen significantly since late 2021, the fact that millions of people have tried and successfully used on-chain and decentralized applications means that there is tangible value in running things on chain. This leads me to another reason to be optimistic:

Reason #2: Building is here to stay (and grow)

The second reason why I’m excited about crypto moving forward is that building does not seem to slow down. Here’s an example:

This graph shows that the number of downloads for ethers, a widely-used library for Ethereum dapp development, is continuing to grow despite the significant slowdown in market activity during the past months. This increase in developer activity is mirrored by all other major libraries and ecosystems (@solana/web3js, @walletconnect/client, etc.).

Here are some things I’m excited about:

• Social protocols like DeSo, Farcaster, and Lens, which are building increasingly vibrant decentralized social media.

• Apps like Mad Realities, MintKudos, Station, CyberConnect, and Utopia Labs continue to build applications for new and innovative use cases derived from crypto’s core use cases.

• Crypto games are seeing serious builder activity too. During the past year, some of the sharpest people I’ve met are working on on-chain games, including teams from 0xPARC, Curio, Critterz (famous for adding NFTs successfully to Minecraft, who are now working on Crypto Towns).

• Infrastructure providers that help developers build better and more user friendly applications easier, like Sardine, Spindl, Coherent, and of course ourselves (Marble Wallet).

Empirically, it seems like the amount of effort that is going into building high quality crypto applications is still increasing, even though prices have tanked. This is a very good sign that the industry is still in an early stage of growth and that there is still a lot of value to be created.

Reason #3: Less noise means more opportunity to build products with real use cases

Many crypto founders who care about building real businesses that serve and make real users happy often felt alienated during the past two years. Too often it felt like the fundraising environment rewarded those who could show the biggest token price increase in the shortest amount of time, often through shady or borderline illegal methods.

This market crash and pause in retail demand presents an opportunity for those who want to focus on building products that create value. While this may also create headwinds, since crypto adoption has been traditionally correlated with increasing prices (and it would be foolish to think otherwise), I believe this is also an opportunity to build apps that make use of crypto in ways that is not directly tied to immediate growth in token prices.

There are some use cases already that show how crypto rails can provide value in the real world. For example, Reddit launched an NFT collectibles program that paid somewhere around $110,000-$270,000 (depending on how you estimate it) to 30 or so creators in the first few months of existence. This is a pretty impressive take rate for creators compared to platforms like YouTube/TikTok/Instagram, where the platforms capture 100% of the profits that result from creators’ work [1]. Furthermore, most users on Reddit don’t even know they’re using blockchain infrastructure when trading those collectibles, which is a further win for crypto.

Another example is the world of decentralized social networks. While still small and niche, protocols like DeSo, Farcaster, and Lens are seeing budding communities starting to grow on them. For example, Farcaster sees about 1.5k weekly active casters (2.9k MAUs), according to their open source stats dashboard. DeSo sees even more than that: they have between 6 and 10k DAUs according to their dashboard. This is a really positive development for those who are excited about building social networks on decentralized stacks with competing clients.

Last but not least, Dark Forest has shown that a crypto game can be immensely popular without relying on speculation and play-to-earn schemes. Dark Forest’s success has inspired some of the smartest people I know to spend time on building the next generation of gaming (which takes advantage of the interoperability and openness that comes along with being on-chain). Since big things often start out looking like toys (think about the Homebrew Computer Club), I’m very optimistic about these projects.

The question that many builders are facing right now is whether what they’re working on makes sense in an environment where retail demand for pure speculation is dead for a few years. I expect many will say no. However, out of the ones that say yes, I expect to see a higher quality of applications than what we’ve seen so far, since they’re subject to the ultimate capitalistic force: making something people want.

While token prices can help retail get onboarded into crypto (mostly for the promise of speculative gain), they can also mask the lack of inherent PMF. Low prices are bound to create a higher bar for success than what we’ve seen during the past few years. I expect that the builders who figure out interesting things to build and do before the next bull run will do very well.

How we’re navigating the next few quarters

While there are some strong reasons to be optimistic, things look pretty bleak overall moving forward (both for crypto and the tech world as a whole). A few things are clear:

• Fundraising is going to get a lot tougher compared to the past two years.

• Retail demand is paused.

• Increased regulatory oversight is coming.

• High rates and fears of recession are a unique new challenge for crypto (none of the past bull runs happened in an environment without quantitative easing).

• A lot of trust will have to be earned back by the crypto community.

Because of this, the checklist of exercises (a slightly modified version of Elad Gil’s) we’re reviewing and paying attention to moving forward is:

• Scenario Modeling: Do our plans for the next few quarters make sense if asset prices continue to fall and we’re on the brink of a recession? How does this affect headcount growth, cash needs, etc.

• Headcount Review: Given the scenarios above, what would postponing hiring for longer look like (given that we don’t really have that many employees to begin with)? Can we really justify adding more people? If so, what are the milestones we’d want to see before we grow our team?

• Review projects and assumptions: Are there any GTM or EPD projects that don’t really make sense anymore? What can we do to try getting to PMF instead? Are there new opportunities that are now more viable due to less noise?

• Free Round: Let’s assume that part of the last round of funding was “free” as a result of market conditions. If we subtract that money, what would we do differently, if anything? Are there any expenses we wouldn’t spend money on or defer until we hit certain milestones?

• Culture and expectations: What is the right company culture and mission given the past few months? Are we well positioned for what’s coming next? What are the expectations we set for ourselves during the “crypto winter”? What are the signals we want to rely on to stay optimistic? Are we still bullish on being in-person?

• M&A: Should we sell the company and/or the limited IP we have? If so, to whom? (I don’t think this really makes sense for us, but we’re trying to cover all possible scenarios!)

Building in a bear market looks scary, especially to people who are ambitious and are impatient. However, given the optimistic reasons I mentioned above, I think there’s still a lot of opportunity in crypto left for those who stay. As you can see, none of the reasons are tied to why FTX collapsed (or any of the big scandals this year). As long as we stay lean and focus on core product-market fit, I think we’re maximizing our chances of winning. That being said, it’s also likely that things can take a lot longer to recover, which will also slow us (and everyone else) down indirectly. However, winning in face of that adversity is a bet we’re wiling to take.

Thanks to John Suh, Nader Al-Naji, Balaji Srinivasan, Diogo Mónica, Antonio Garcia-Martinez, Ron Miasnik, and Dan Romero for reading and commenting on various drafts of this post.

Subscribe now