computational notebooks - aaaaaa

You have probably heard about computational notebooks, or just "notebooks" — it's cleaner.

Maybe you're like me. You were hardstuck Gold 3 in Season Four, cutting Russian Literature class, your parents get mad, they send you to a boarding school in the UK. It's fine.

After two years of being a good chap you get your braces off, get an A* in Geography GCSE where you have to write a 300-word "essay" on how a local car park was renovated — it is now time to decide what you want to do for the rest of your life.

So you search: "university degrees salary". Lawyer — no, doctor — leave it to the pros... wait a second, I didn't spend all that time hardstuck in G3 for nothing! You pick computer science and luckily you kinda like it.

And you don't even know yet that the best is yet to come! When you don't have to maintain a list of places that sell beer (ordered by price) in your head, you free up energy for actual useful things! Like PEP8, concurrency vs parallelism, locking, and all the other fantastic stuff that nobody knows what the hell it is.

Before that though, there is an ML module taught by Dr. Carl Henrik Ek, which was brilliant. Despite being hungover you get to learn about the chad version of ML that is statistical modelling and you have to do a lab in... you guessed it, a Jupyter Notebook.

history

Jupyter is not the first notebook but it is the notebook. Fernando Perez (scientist and the creator of IPython that became the Jupyter Project) once shared this cool history before redirecting the link to something else. But you can't keep away from me, Fernando.

In that history it becomes clear that a computational notebook as envisioned by Jupyter was geared primarily at scientists, not the myriad other use cases like education, data science, data engineering, and prototyping that it came to be useful for.

Notebooks in general are not built for small to mid-size data team user groups; they are educational tools and tools for doing science. There may be an underserved point in the design space as smarter people than me have highlighted.

Products like Hex enjoy dunking on the common issues: scale, state, and sharing.

The scale problem boils down to "I can't SELECT * from my BigQuery table because it will blow up my RAM". I am not sure about this one, but I don't know anything.

State is about traditional notebooks allowing out-of-order execution of cells. Perez mentions this in his history as more of a feature than a bug. From what I can tell, this execution model pleases as many people as it frustrates.

And sharing, where Hex conflates versioning and presenting to stakeholders. Assuming you use git, versioning notebooks is not straightforward — the serialization format is whatever nbconvert spits out and Jupyter serializes the outputs by default so your git diffs can quickly get weird.

But then again, nobody out there in the real world even knows what git is; they're still emailing each other Presentation_copy_copy_123_final_copy.pptx.

Some notebook products deploy as apps you can share with stakeholders. It feels like it all depends on the serialization format and the standard library. Marimo notebooks are just files that run in either edit or run mode, Verdant notebooks version every keystroke in every cell. World's your oyster.

If you imagine a multi-dimensional valley of all possible literate programming implementations, Jupyter, Hex, Verdant and others are hills in that valley. Or they are little houses on the slopes of the same hill — I dunno, I don't go outside.

You can tell the implementations over the last few years attempt to climb the hill. Not so long ago, an asteroid crashed into the side of the hill and it has the letters A and I written in big red letters (in Inter) on its side.

To keep with the asteroid metaphor, parts of this have landed in numerous other valleys and will doubtless impact the ongoing climbs or even catapult the climber to a wonderful new hill (or valley!) to explore. You may not agree, but I don't care — this is my blog.

worse is better

Richard P. Gabriel once ~shitposted~ mused that "worse is better" talking about AI (back in the day version). The jury is still out on whether the principle holds.

TLDR: "Worse is Better" argues that simple, pragmatic C-style design wins over elegant Lisp-style perfection because good-enough software that ships beats perfect software that doesn't.

In mid-2015 after Java overtook C with its more restrictive (the right thing) API, since then two other languages — one developed in 10 days (well, the first prototype anyway; the real version took until 1996) and another named after a comedy troupe but everyone thinks it's a reptile — have risen to prominence. This might have more to do with software distribution than with language design.

JavaScript needed a megacorp to step in and write a gradual type system that works like a compiler to make the code stop constantly breaking. This birthed a generation of type gymnasts who spun the typing plates until the Querio repo started taking 1 minute to "compile". MS are now re-writing their TypeScript compiler in Go, promising 10x speedups by mid-2025. Where are they???

Python is widely adopted (and my second-favourite language), but it has this thing called GIL, who is not in fact your grumpy neighbour from downstairs but a grumpy design choice making parallel execution of Python impossible for CPU-bound tasks because only one thread can execute Python bytecode at a time — a mutex that's been the bane of performance for people smarter than me for decades. Until they fixed it recently.

It is clear to this reporter that worse is better in the short term but the inertia flows towards better is better. It is not uncommon for the "better" version to exist in parallel (TypeScript) only to then start climbing towards a new local optimum — sometimes even requiring a complete rewrite in a different language entirely.

Ah yes you wanted to talk about notebooks? Fine...

what's good about notebooks

You have your code and your docs and stuff all in the same place — it's really nice to see everyone being friendly like that and potentially taps into other benefits that writing may have on the clarity of thought and ultimately the usefulness of the science that the thought generates.

what's bad about notebooks

This study highlights the most common errors in Jupyter notebooks:

• Kernel crashes — user has to restart, potentially re-running expensive queries because the data lives in the kernel process.

• Environment and Settings — user cannot use a notebook due to environment issues (imho this is downstream from Python in general until recently).

• Conversion and portability — user cannot share a notebook with people because the conversion code broke.

• Connection bugs — users often want to pull data from external data sources and their connection configuration is not portable across environments, usually due to infrastructure constraints on the target hardware (firewalls, etc.).

• Cell Defects — cell outputs do not look or act as expected.

• Processing — getting MemoryError, etc. as alluded to above.

• Implementation — someone wrote buggy code and/or they are not sure how to do what they are trying to do with the tools they have access to.

By far the most common classes of bugs are Environment and Settings (StackOverflow — 43.2% | GitHub — 35.6%) and Implementation (StackOverflow — 22% | GitHub — 44.2%).

For me as a certified software engineer, a lot of the above amounts to skill issues. But as we discussed, the critical user mass rightly expects the notebook to handle these issues for them.

The paper adds an interesting dimension to each issue class — the average annual growth. Environment and Settings and Implementation lead the charge as expected at respectively 38% and 48% annual growth rate on average.

Interestingly, the Conversion and portability and Cell Defect issues grow at a higher rate than average, suggesting that users are trying to share their notebooks more often than before and are running into issues.

The paper recommends some features for future notebook implementations:

Nothing new there...

The paper harks at a familiar tension around notebooks. The tension comes from the fact that many actual Jupyter users do not come from the background the tool expects. Because of Jupyter's widespread adoption, other products can be reluctant to address these issues because they often target the same user group.

If it hasn't become clear already.

jupyter is complicated

Even at a glance, 5 socket connections per kernel (read: per-notebook) is a lot of complexity to manage if I just want to run some code in isolated environments and get access to the outputs. But then there is the jupyter-kernel-gateway.

There is a decade-old shell channel that defines an execution pipeline that boils down to an exec in a UserNamespace in a Python process behind the ZMQ (or gateway) broker. It also supports magic strings and debugging, but that's not important right now.

The age of this architecture should imply the baggage and direct us to look at other approaches to process isolation that have become possible in the last decade.

So I thought: "screw this, I'll build my own — I am a genius anyway". So I git cloned every OSS notebook product under the sun. Read the first one... Claude Coded the first one... stepped through the first one in the debugger... I kind of understand the first one now. It has been a week. I will share when I know more.

notebooks at Querio

Here at Querio we are nothing if not knowledgeable about everything. We see that teams in and around our ICP reach for one notebook or another for their ETL, dashboards and insight generation. We also want to serve the other side of that equation — the internal customers of the respective data teams that commission the ETLs, dashboards and insights. This is what Querio offers.

Our agent uses a notebook under the hood as the interface between our customer's data; it runs queries and generates insights.

Every insight you see in Querio can today be traced back to a cell in a Jupyter notebook. This is really nice and exciting for us because we can open up the notebooks to the other side of the stakeholder/data-team boundary and be left with, well, a notebook with a (very high altitude) LLM integration.

So we were gonna do that until we realised our current wrapper around the Jupyter kernel is ripping at the seams under some of our customers' workloads. This is what motivated the analysis — the first artifact of which you are reading.

We need some more time, but we might have something cool to share soon. Well, at least I hope I think it's cool when we think of it.

future of notebooks

Going back to all the different dimensions of notebooks as a product. At Querio we think there is a future where none of this malarkey matters at all. All everyone wants is answers to their questions, underpinned by rigorous analysis that makes the answers trustworthy (assuming you trust the science behind the analysis).

The stochastic token-mulchers are getting pretty close. Given some context, you can get answers to common questions, but sometimes a person of science must step in and help untangle the tokenistic bonanza to get to the right answer, or even do some good-old data exploration (science) to ask the token-mulcher better questions.

The natural place for the data analyst and Shoggoth the token-mulcher to meet may be a computational notebook. Or it may be something else entirely.