Artificial Science

When I was first introduced to computer science with the classic Lego Mindstorms NXT robots in 2009, I was entering a very ordered, logical world. Of course, once set loose to explore the real world, our primitive robots would be disturbed by many uncontrolled influences. However, on the computer itself you could, if you wanted to and with enough time, go down to individual currents running through wires to debug the programs running on them. Digitalization enabled precision in our continuous¹ natural world. In fact, I soon was very eager to discover how a computer actually worked and went down the rabbit hole of learning how to build simple CPUs in Minecraft² and later Logisim³. At some point, it all came together for me and I had the feeling of exactly knowing what was going on down to the bit, at least for simple programs and architectures. When in doubt, a manual told you everything you needed to know. Later, when I started studying computer science in 2017, theory came into the picture as well, enabling verification of programs and the like. Computer science felt like the perfect deterministic world. Perfect by human construction, enabling reasoning about every bit and its effects.

Artificial Intelligence and the Departure from the Realm of Bits

Deep learning is a rather ironic part of computer science, which concerns building (by educated guessing and rules of thumb) a complex system using continuous calculations on top of the digital, i.e., discrete, computer.

And then came my first closer contact with the artificial intelligence (AI) subfield, a field that had long caught my attention and that deeply fascinated me. In particular its machine learning and deep learning parts were very popular upon my first contact. In hindsight, deep learning is exotic in the context of the computer science I started with: instead of constructing the relation between input and output (the program) from scratch, it is constructed by the computer itself, by walking through a continuous, high-dimensional landscape of possible relations. One can no longer directly control the outcome, only indirectly by constraining and shaping an optimization landscape with inductive bias. Even worse, classical debugging is completely powerless: one can still go down to every bit, but the direct relation between bits and behavior escapes our human comprehension. Generally, constructing and debugging such a system involves a lot of trial and error, magical constants, and rules of thumb.

One could view this situation as though the program runs on a non-imperative virtual machine (an artificial neural network). On this machine, we can go through the running program step-by-step, or layer-by-layer, but there is no directly apparent semantics or interpretation, at least not with respect to the program’s high-level purpose. With enough time, one could maybe understand a particular neural network in a lot of detail. However, this effort would be mostly futile, as new versions are developed constantly and the findings would largely not carry over to the next version. In contrast, new versions of CPUs can mostly be understood with knowledge of the previous version (or the manual). So deep learning is a rather ironic part of computer science, which concerns building (by educated guessing and rules of thumb) a complex system using continuous calculations on top of the digital, i.e., discrete, computer. On the other hand, it allows solving problems that are typically out of reach for classical programs, where there is no, or not yet, a convenient way to express the solution as an algorithm.

To complement this success, the field of “explainable AI” is actively working on adequate solutions to the interpretability problem. At this point, also note that there are other parts of computer science that are historically experiment-driven and that do not solely rely on algorithmic analysis. One notable example is high-level performance measurement, which is influenced by many (stochastic) factors. It requires replications, controlling for warm-up periods, etc. Also, there are other fields that model complex systems on top of digital computers, such as modeling and simulation. Actually, as computer science and software engineering progress, stacking abstraction levels leads to problems with properly characterizing every possible behavior of a program right from the beginning. So this always has been part of the “science” done on and with computers. However, deep learning takes this to the extreme by relinquishing the mechanistic control, i.e., the ability of humans to purposefully alter the program. There is no human design or manual left to refer to. This situation requires different methods than “classical” computer science. It has been acknowledged in the past. And this trend will continue even further. In this sense, below I explore how the arising ubiquity of research on AI, but also with AI, in many parts of computer science leads to the term “artificial science”.

Natural Language as Interface

A key enabler of the success of LLMs is that these models allow using computers based on natural language, rather than formal languages, such as programming languages or command-line interfaces. The interface is now much more convenient and accessible to a broader audience, but also less precise, and its inner workings are elusive.

In fact, deep learning is as close to natural science as it is to computer science or mathematics. And recently, it made another step towards the former. Machine learning systems were long slowly gaining traction in our daily lives, accomplishing specialized tasks, such as image classification. In 2022⁴, however, they started to convince⁵ humans with the quality of their talking, drawing, and audio synthesis. This led to a broad public interest, followed by massive investments of money and research into such generative AI (GenAI) models. Of particular interest are large language models (LLMs). Their foundation is a system to predict the most probable completion of texts. By fine-tuning them on conversational data and on human feedback to output a two-person chat structure, they can be formed into chat bots following instructions. These systems are on a completely different scale than previous deep learning models, going from millions to now hundreds of billions of parameters, exhibiting vast modes of emergent behavior. They are the next step on the path away from the classical mathematically-driven computer science towards a more empirical approach, in terms of the kind of research we do, but also in terms of how computers are used in general.

A key enabler of the success of LLMs is that these models allow using computers based on natural language, rather than formal languages, such as programming languages or command-line interfaces. The interface is now much more convenient and accessible to a broader audience, but also less precise, and its inner workings are elusive. This also enables a more experimental approach to software development, where software is written based on imprecise instructions provided in natural language. Through pipelines of several LLM “agents” with different instructions and automated testing, a final software product is created without requiring a human to write a single line of code. Humans are taking the role of the coordinator, providing high-level guidance using their native form of communication. One can also implement reusable sets of instructions, so-called “skills”, which an LLM agent can discover and use on-demand. Writing such LLM “programs” is a very interesting experience. On the one hand, it feels a bit like science fiction and very convenient to write an “algorithm” using natural language. This is also much more flexible than a standard program. On the other hand, it requires some trial and error to get the LLM to behave as one intended (on average or in 99% of cases), as the language leaves room for interpretation and also needs to be tailored to the unique behavior or capabilities of the LLM at hand.

Rebranding

The current AI wave will require and might permanently anchor empirical methods in computer science. Computer scientists will have to acquire and use skills beyond “classical” computer science more than ever before.

Current GenAI models and the infrastructure built around them present such a big leap in terms of capabilities, that they managed to claim the term AI for themselves in the public discourse. This term is typically used to refer to the study of intelligent systems in computer science as a whole, where GenAI strictly speaking occupies a small fraction of the design space. Hence, such a branding implies that they are able to cover a much broader spectrum of intelligent and adaptive behavior, such as language comprehension and reasoning, than any other method in AI. A more fitting term often used to refer to such fully generalist systems is artificial general intelligence (AGI). It is currently vividly debated to what extent, e.g., logical reasoning may arise from training on natural language and emulating thought processes in natural language.

Now, as a curious PhD student, I obviously followed the trend and did some reading, exploration, and coding myself. After a while, it occurred to me that GenAI requires more empirical study than probably any human-made system before it. They are of course interesting as a natural language interface, but also in many other regards that are not directly obvious. For example, they can basically be seen as a content-addressable compression of human knowledge: As their distribution over possible continuations is trained on large parts of the text contained in the internet, it is interesting to explore their contained biases, for example toward German federal states. They can also be seen as simulators of humans and are tested as replacements for human participants in studies or surveys. Further, we can try to condense their human-like or intelligent behavior into mechanistic formulas by novel symbolic regression methods, for example for crowd simulations. And the list continues. Working with GenAI models in such ways is very empirical and involves long-running experiments and delayed results. The current AI wave will require and might permanently anchor empirical methods in computer science. Computer scientists will have to acquire and use skills beyond “classical” computer science more than ever before. These skills are familiar from my work in modeling and simulation, where we perform (virtual) experiments, deal with stochasticity, and perform statistical analyses and tests.

Artificial Science

We are no longer constructing artificial systems by hand, we are studying the results of our creation and creation itself is automated. We might need to start an artificial science.

This new thing we do with our computers is no longer driven by theory, not even mechanisms, but by pure experimentation. Experimentation much like that on complex (living) systems in natural sciences, which evolved over billions of years and are now there for us to reverse engineer. The evolution of AI systems was initiated by humans, but the result is so complex we can no longer hope to understand it without specialized abstractions. Curiously, unlike biologists, we even have complete access to every weight and parameter of our specimen. Yet, this does not directly buy us understanding. In a growing part of computer science, we are no longer constructing artificial systems by hand, we are studying the results of our creation and creation itself is automated. We might need to start an artificial science. It feels like AI and other computer science fields, such as software engineering, are pivoting away from the perfect world of “classical” computer science toward a novel direction that is worthy of a distinct name. Rather than a completely new discipline, it is one born out of existing efforts like explainable AI and the increasing need to study AI systems with custom and proper methods. In fact, while writing this text I discovered that, already in 1969, Herbert A. Simon used the term “artificial science” in his book “The Sciences of the Artificial”. In using this term, he mostly argued for the design of systems by humans to be studied, which often still fall in the category of systems with a manual. Still, even in these early days, it was apparent that artificial systems deserved and required empirical study, although back then symbolic systems were the state of the art in AI. Also in the era of connectionist AI, some authors argued for studying machine behavior as an interdisciplinary field before the advent of the GenAI hype. While this opens up the opportunity for young researchers to shape new foundations, one has to get accustomed to a shift in paradigm.

What will artificial science bring? Of course, explainable AI will continue to grow in its importance, also as GenAI systems are increasingly prominent in humans’ daily lives. In the absence of a new AI winter, we may also see more methods from the natural sciences being adapted to explore these new artificial systems. Research will be done on finding adequate tools to measure and summarize the AI system’s “inside” and on finding models (abstractions) able to describe high-level behavior of current AI models. Their “emotions”. Their psychology. Their statistical modes. Computer scientists need to learn how to properly document experiments, write lab books, devise and refute hypotheses, and control for external influences. They need to regard AI models as the opaque stochastic systems they are and perform experiments with replications, convergence tests, and analyze distributions of results. This is not only true for researching AI systems on a low level, but also on higher levels, e.g., when writing skills and prompts. It is true for researchers in the field of AI, but also other parts of computer science and beyond that want to study it from their perspective. I have seen several papers just prompting large language models, reporting the results, and claiming their system prompt to be a “method” or the AI system to possess certain capabilities. Indeed, such reports are a good way of showcasing the capabilities of current AI systems, but are prone to several criticisms from a scientific standpoint: What is the variance in the answers? What about changes in one of the many hyperparameters? What would be the influence of slight changes to the system prompt? etc. For the very large “frontier” LLMs, answers to very common questions are rather stable, as sufficient training data made the probability distribution over completions narrow enough. However, in general these variances may lead to the method failing when used beyond the given examples. A rigorous approach is very time-consuming⁶, but is required to ensure reproducibility of results. Of course, as outlined above, this has always been part of machine learning and computer science, but from what I have seen it is still not acknowledged widely enough. I presume there is still much to be learned from the natural sciences and other disciplines. In any case, the empirical study of the artificial is now very prominent and quickly gaining importance.

Conclusion

It is just as exciting as it is daunting to live in our times and be part of this process (or revolution?) in some way. I can talk to my computer! Isn’t this all amazing? Are skills like programming, writing, and using the Linux command line I spent so much time learning now suddenly obsolete or not appreciated anymore? People tend to chat with an AI bot instead of asking me for help with Linux questions. Might it come for my job as well? It might be that, for my generation, the current AI hype is what the internet was to the young adults in the 1990s. The implications are also very similar, spanning many risks, but also opportunities. Maybe some of that digital deterministic goodness I once started with will survive. Or maybe we will soon fully rely on fuzzy natural language interfaces and solutions⁷. In any case, some comfort can be taken in the fact that studying itself will continue. Rather, what and how we study may change, as it always has.

Epilogue: Some Science Fiction

While we might not see positronic brains any time soon, there is certainly room for AI mathematicians, AI (hardware) engineers, and also AI psychologists.

Interacting with, actually talking to, instructing, and “programming” AI agents with natural language feels like science fiction is becoming reality. In fact, also in combination with the ongoing and recently renewed interest in humanoid robotics, the current developments are not too far from a classical science fiction scenario once (ca. 1950) described by Isaac Asimov in his Robot stories (“I, Robot”, “The Caves of Steel”, etc.). Therein, humans found a way to construct “positronic brains”, a very delicate and advanced technology to store and enact self-aware intelligence (they don’t necessarily have to be paired with robots). In the novels, there is a whole range of jobs and disciplines involved in their construction and study. While we might not see positronic brains any time soon, there is certainly room for AI mathematicians, AI (hardware) engineers, and also AI psychologists, empirically analyzing our new creation from every possible angle and building models of their behavior: an ODE system describing changes in emotion based on inputs, identifying points of instability in personality, etc. Indeed, when reading how, e.g., the company Anthropic reports and writes about their experiments, we can already see elements of this. Going even further, in the long term, I would not be surprised to see a comeback of analog computers specialized to AI. For example, recent preliminary results show how in-context learning can emerge from chemical reaction networks. While the current hype is obviously enabled by exploiting available digital hardware, the inability to reason about an AI system’s behavior from the ground up and its non-deterministic behavior remove a basic motivation behind the digital computer. In other words, if we are left with researching these systems experimentally anyway, the difference between an analog and a digital foundation diminishes and which to employ becomes a question of computational and energy efficiency (and maybe also how “fiddly” it is to build and deal with, but current AI may help with that part). Of course, this is still in the category of science fiction.

I have always been wondering whether science fiction is a precursor of any new direction in which humanity progresses or is sometimes just very good at prediction. For example, the short story “Slow Tuesday Night” about an ad absurdum accelerated world seems to have predicted LLMs half a century before they became reality (see the part of the story about Maxwell Mouser). If LLMs really do manage to surpass humans and drastically accelerate our world, I am wondering whether we are able to dream up new ideas fast enough for either statement to hold.

Home Imprint Privacy

Opinions shared here are my own and do not necessarily reflect the views of my employer or scientific collaborators.


learn about Buttons

Copyright (c) 2025-2026 Justin Kreikemeyer
Licensed under
CC BY-NC-SA 4.0