[nybble-engine-toolZ home] - [A radical meta-art system shooter] -  [Narratives of the code]

Mediapoiesis, autopoietic compiling and code-critique in the technological narrative
Narratives of the code - A research desideratum exemplified by network-poiesis

Margarete Jahrmann

“The re-engineering of existing, commercial systems (such as game engines) or their inversion and subversion has also increased, although this territory arguably remains under-explored. Considering the potential of the digital medium, there are still relatively few works that create open systems by allowing users a sophisticated reconfiguration or rewriting of the system itself or by relying on networked communication processes in challenging ways.” (Prix Ars Electronica 2003, Interactive Arts, jury statement) 1)

In the context of the given topic of Networked Narratives, the network game and installation *nybble-engine-toolZ* (by Margarete Jahrmann and Max Moswitzer, http://www.climax.at) will provide an illustrative example for a mediapoietic work. 2) In the following it becomes clear, that this work is not just interactive but even more so a media narrative made out of codes, network protocols and communication. But first we will have to clarify the immanent specifics of such a technological narrative. While looking in more detail at this practical example, it will be easier to define the meaning of mediapoiesis, especially as this term still has to be further developed. 

*nybble-engine-toolZ* - A radical meta-art system shooter and practical model of mediapoiesis

A so called shooter defines an established type of computer game which is based on the process of sighting or detecting and shooting an object. The project *nybble engine toolZ* re-engineers such an existing commercial system as a game engine sprinkled with network commands. Game engines are operating software for real-time graphical rendering, which takes place in predefined environments, although the way of rendering the image is depending on certain actions, events and triggers. Often it is also the basis for a multi-player environment and for a network protocol. A nybble is the unit of half a byte or four bits and thus the basis of every digital conversion. As the numeric equivalent of the binary code, it exemplifies the internal logic of a software, that converts codes and protocols on a server into various representations.

*nybble-engine-toolZ* - as a radical meta-art shooter, serves as a self-ironic multi-player statement tool. The network suggested for the interactive installation consists of players on site in the installation, actors on a remote site in the city and spectators connected to the running nybble-engine game server via the Internet. When shifting between these positions participants become <spect-actors>, active spectators, changing between the interactive and the interpassive role of a player with code and a writer of code, between a visitor in an installation and an actor in a virtual environment. E-mails can be sent into the nybble-game environment and change it. There are also outgoing mails to president@whitehouse.gov. We applied the newly coded game-features, as out and ingoing mails, sensible from a <cyber-Nethical> point of view. The scientist and philosopher Heinz von Foerster introduced the term KybernEthik, (Foerster, Berlin 1993). He is talking about a second order ethics, according to a second order cybernetics, which is following the observation of the observation as principle. In intended contrast to the violent aesthetics of the computer game, the project is based on anti-war e-mails, commissioned with each shot made with the game pad in the installation and the online network. So the toolz are usefull for the lessons in self defense, which the citicens of contemporary democratic societies should take to be able to resist against manipulation and control, as Noam Chomsky described it recently in his book Media Control, (Chomsky, New York 2002). Each toolz-email is displayed in real-time as both ASCII text and newly generated visual object. Out of this visually coded environment, text messages are sent. This can also be commands from the running engine itself to the network. E-mails inside the engine trigger the environment of action-bots (these are software player-robots with a minimal artificial intelligence), although one can also directly log in on the game-server. While moving through the environment, trace routes are started from the game to a number of crucial government servers. Network activities outside the engine are displayed in the game environment in real-time as texts and 3-D objects.

By reversing the effects of a specific action, such as jump’n run or shoot, the rules of the game engine are reversed: the unreal tournament becomes a situationist détournement, an inversion. Attack is collaboration, shoot is communication, and playing becomes the editing of code!

The installation: an interactive/interpassive group experiment 

The architecture of the installation room, where the participants are able to take seat, is shaped like the 3-D form of the virtual environment. This form is a trace of the movements inside the network  environment. We transfer this data-form into the installation room, by making a 3D printout in real size of the virtual object. Following contemporary architect and theorist Bernard Cache’s example, we call these forms objectiles. In reference to Bernard Cache, Gilles Deleuze defines an ‘objectile’ as “a very modern conception of the technological object: it refers neither to the beginnings of the industrial era nor to the idea of the standard that still upheld a semblance of essence and imposed a law of constancy (…) but to our current state of things, where fluctuation of the norm replaces the permanence of a law; where the object assumes a place in a continuum by variation (…) the object here is manneristic, not essentializing: it becomes an event.” 3)

Inside the virtual space each actionbot or player’s avatar carries a data object instead of a weapon. As these objects represent command lines and processes from the game environment, they also appear as a part of the installation interface, as laser-sintered <objectile>. In its pure form as code equivalent and as discourse object it is exhibited as a code hardcopy. This freeze frame functions as the art-coded interface to the data-world in the installation.

The peer-to-peer software of the installation converts network processes into three-dimensional abstract movies and projects these onto a 180-degree screen, such as experienced in an interpassive cinematic situation. The network codes and commands are generated with the aid of game engines into audio-visual so called machinima movies. Simultaneously they are playable command lines in the networked game environment.

The participants in the experiment can, however, also become active in the installation space. At a small control desk they see their individual view of the navigation, which presents a subjective perspective at variance with the server view on the screen. The players use an ordinary game pad to log onto the network of the installation and to enter into the shooter environment, where projectiles of data object(ile)s, action bots as well as other players are flying about. Each time a data object is hit, network processes are triggered, each time a shot is fired with the game-pad, an anti-war email is sent. There are two choices for the spectator: either to become a player who concentrates on the small Gameboy monitor and successfully navigates and influences what is happening on the big screen, or to be the one whose attention fluctuates and combines the different perspectives – those of man on the small screen and those of the server-machine on the big surround screen . If one concentrates on the personal view, it influences the entire picture being simultaneously generated and projected. Seeing this simultaneous view of one running application in different windows might motivate viewers to intervene in the communication. In the case that no one is taking part in the installation space the action bots and online players take control over the action.

Mediapoietic thinking

Our cognitive interest lies in the rise of separate display formats in man-machine perturbations on the level of group interactions, which are not predictable just by the programming. The applications’ volumes form the central topics, following Felix Guattari’s notion of the machinic, but recombined, according to arising contemporary modding cultures (modding is a term that stands for modifications of software/code classes in game engines as well as individual modifications of hardware). The definition of an engine as an algorithmic machine is the discursive starting point. Software texts as game engines are reflected and the work focuses on the differentiations and theories of the machinic towards the actual status of the machine as an engine in coding-cultures. Machines and engines are here not considered as tools but as media, as modification material for the arts. In the interrelation of different media layers, the application as part of the complete engine becomes a tool run by action-bots, interpreted as non-linear software and by human players in mutual exchange. What mainly matters are the man-machine and machine-machine effects of communication and their circular and causal feedback mechanisms in the techno-social system environment: as broadcast messages inside the live generated and visually coded environment display real time network activities from outside the engine, and as commands can also be sent out of the running engine into the network - the same time a ping to a gov.server is started by pressing another game pad button - then the users construct their environment, collaborate with the play.

In such a networked shooter group experiment players take over the role of humans, action bots and server-protocols themselves. A network application forms the experimental set up as a modification of an existing game engine, yet not a game in the common sense, although there are remaining references to game-culture and history. The artistic challenge of this work is represented in the custom coded engine as an algorithmic machine for real time rendering of user activities but even more so for audiovisual networking. Such a redefinition of game engines promises from our point of view the possibility of browser free networks. In the actual case the usually hermetic engine is opened up by integrating the layers of the network as material for its aesthetic surface. A new hermetic system is shaped then in the technological sense that such an engine uses its own proprietary network protocol. In the nybble-toolZ these protocols are merged with open code standards.

The technological narrative

The main inspiration derives from the structure of the computer game application, but its initial narrative is based on the technological potential. A network protocol based on game engines is telling a story of optional networking as an immersive environment. It can be used on a console and it can be combined with so called real worlds when it is implemented on combinatory interfaces, which are superimposing additional information to the real life image through data goggles. This was investigated in an earlier experimental project based on a Quake version in 2002: 4) The School of Computer and Information Sciences at the University of South Australia transferred the campus into an augmented reality map. Players are wearing a semitransparent data goggle while walking on the real campus and get additional information, or monsters, superimposed. The style of game navigation and interaction is completely maintained, also with selfbuilt shooting interfaces. While looking at this experimental application one misses any change of metaphors or surface language of the original shooter game. The actual change takes only place on the code level. In contrary the game is transferred out from the network onto a level between the technological and the real life reality.

When a game engine is merged with open codes and standards it can be a potential tool for system operating. This was implemented 1999 in a shooter version, where people were running around in a multi-user environment and shooting processes. It was based on a version of Doom as a tool for system administration. As I was listening to Anil talk about demons spawning processes and sys-admins killing them, I thought, ‘What a great user interface!’ Imagine running around with a shotgun blowing away your demons and processes, never needing to type ‘kill - 9’ again. (…) I downloaded one of the many versions and added a few lines of code that would spawn a new soldier for each process, renew the process when it is wounded, and kill the process when it dies”. 5)

These kinds of narratives are embedded in an existing technology on one hand and game culture on the other hand. Although the mentioned applications are sticking to the syntax and images of the language of shooter games, they are telling a kind of neutral story. The technological reality is made of the real live circumstances of individuals creating these software modifications, the network gamers. They are not quitting the aesthetics or the terminology of game culture but are providing a different idea of interface culture. This is the second big narrative shaping up in the field of game inspired technologies: the techno-cultures itself are telling the stories and therefore underlining the significance of computer games as a cultural phenomenon. 6) So the basis for a networked mediapoiesis is both, the techno code and the cultural application of code.

These particular types of networked narratives produce its very own computer literacy. The player community gains interest, when the gamer clans are originating their own machine movies, their individual machinimas (as described above.). This means the game users become pro-active, they start to modify, to recombine. But here the dispositive raised by this term is that of a community, a culture, or a so-called scene. It is defined by the act of dealing with programmes, programme languages, applications and codes. During the recent years a complete modding scene developed. It encompasses its own magazines, web pages, discussion groups and a range of minor media, a term that follows Felix Guattari's conception of art and new media towards heterogeneous machines, which are obligatory to link people together. New in this structure of subcultures is the fact that the applications itself are such minor media communication environments. In general modding then exemplifies the ongoing and ever intensifying clash over who will control popular culture. 7)

Autopoietic compiling - pay attention to the code behind the screen!

The central issue now would be, how to treat a question as discursive in the context of mediapoiesis as a form for code-narratives. Or in other words, which strategies make a statement on code-reflection discursive. As it wouldn’t make sense to force code literacy, a meta-language on code has to be developed. The socially constituted expectations of networked narratives must be taken into consideration under the aspect of viability. This means that a reflective-theoretical art that claims to preserve complexity must articulate itself in a form that is still identifiable as network code and narrative at the same time. In this way, codepoiesis is processing along the system/environment difference, at its very edge so to speak, where the artistic ability to dissolve and recombine code structures can reproduce self-modifying narratives. It is not the aim of codepoiesis to explore existence in any form for the purpose of attaining insights into artistic means and methods, but rather to discuss questions that relate to the practice and feasibility of network-poiesis and art in its current social context.

Parts of the following reflections on  mediapoiesis as a specific codepoiesis that becomes visible while executed, I have published in the jury statement for Software Art at the Transmediale Berlin 2003, in cooperation with the artists Amy Alexander and David Rokeby. 8)

“Any piece of computer software exists on two levels. On the one hand, it exists as an executing program supporting some sort of activity or generating some sort of experience. On the other, it exists as a text in its own right, a subjective expression of the writer's ideas and an example of one person's struggle to give form and function to an idea in the particular medium/language of executable code. The human reader has an immediate and direct relationship to the 'source text' of a poem or novel, rendering it into an imagined experience using their comprehension of language, their memories and life-experiences. On the other hand, a computer program's text is generally hidden, interpreted by the processor, operating system and hardware before being presented to the audience. But the inaccessibility of the program's source code does not make the code-text itself culturally irrelevant. In the case of executable computer code, some of the interpretive activity is shifted away from the human into the machine/code system. This combination of human-written code and human-designed machine is a complicated expressive mechanism that cannot simply be reduced to a function of its surface effects. As poetry and literature can be understood as struggles with the mechanism of a particular language and the limits of language itself, software art on this reflexive level can be seen as a struggle with the mechanisms of a computer language and the acts of quantifying, encoding and logical processing. This engagement is a central yet deeply hidden and under-examined activity in contemporary culture; its importance is amplified by its invisibility. The particular way that an executing algorithm hooks into its cultural context is key to the understanding of the work. When the code itself is very verbal rather than purely algorithmic, this verbal content may either clarify the concepts behind the work, or constitute an important part of the work itself.

A conceptual framework that allows one to discuss software related issues without descending to implementation specifics would be a valuable tool. But reading and understanding the code text itself is not enough. Such a framework or meta-language must not be merely an abstract representation of code function and code structure, but also a way of talking about the ways that coding actions ripple out through executing machinery to affect the social context. An algorithm may be a simple representation of a mathematical or logical equation. But even a simple and easily understood algorithm can be a provocative agent in the complex system of the world. Software defines processes, which reflect and transform the complex world that surrounds them. An algorithm can be thought of as a person's way of making sense out of data. Code transforms the significance of data, and the data that code is applied to changes the significance of the code. They form contexts for each other. Artists and corporations often process the same data through very different kinds of algorithms bringing radically different interpretations of the data. Choosing to put an algorithm into play in the world is an expressive decision that cannot and must not hide behind the apparent objectivity and "innocence" of the algorithm as an isolated logical entity.”

So it gives us control on how to examine the texts of software at large, which normally hide ‘behind the curtain’ of the interface, output and illusions of pure utility or transparency, behind the fiction of 'objective' technology. Looking at software coding as a form of expressive text writing opens up another question: Does an examination of this form of writing offer new perspectives on other forms of text based representation? Software coding exists as one extreme continuum of text writing. Extreme cases often clarify middle positions. Virtually all writing involves processes of formulation, encoding and compression and a deferred but implicit process of interpretation. Software art, while maintaining a consciousness of the interrelationship between the underlying code, the code's functional operation and the cultural ramifications of the code's execution, presents the opportunity to test new conceptional models of language, text and expression.

Code - text - langue - parole - font

To make a code reflection discursive and dialogical, a playable, networked narrative environment has to be introduced. When executed it is at first cryptic. When looking into the visual encryption and listening to the audible codes of the procedural descriptions it soon can be decoded. As the observer, the user becomes the decoder, focussing on the idea that everything is formal text in a software environment. The classes and behaviours of an existing engine are visually transcoded. For this purpose the font serves as a surface. Yet it goes beyond the pure aesthetics of surfaces as it addresses the semiotics of formal languages and their representation in the context of coding.

Re-using a game engine discusses its medial qualities as a programme and as a distributed network text. Determination by text is its focal point. The text of the application is coded as formal language, but can be detected by the programmer, interpreted by the compiler and embodied by the running program through its spatial appearance. In the interface that the executed programme displays a new font can be read. The ‘text’ of space and actors is coded in a generative font and reintegrated into the interface. Surface and code text merge. The formal coding of programme texts in logical and notational systems of programming languages is mirrored in the visualising of text in a new interface culture.

To a certain degree of abstraction, it may also be that trivial and non-trivial machines interact with one another. As the reference to problem orientation already indicates, coded art is not developed by way of analogy. This opens up the option of a functional equivalent that is generated in the reference to software objects and their implications to a dynamic, complex context. This is already achieved through its status of being an articulation in a communication context, where expectations always make up an important factor. These are expectations guided by norms and they are, to a certain extent, dependent on a factual event; in an art discourse, the ‘new paradigm’ stands for a problem construction, to which further constructions - in other words, functional equivalents - are conjoined. Orientation to function tends with appropriate complexity to higher problem specification. This applies not only to the art system as a whole, with all its subsystems, which are structured through orientation to function, but also to the level of art practice, which articulates/focuses/reflects/queries these system/environment differences.

Its functional equivalent relationship, however, does not only refer to the source text. A certain theoretic framing lies in its final evolutionary stage, defining a meta-art system, that serves as a reflective endeavour for observations, in order to observe oneself as a second order observer formulated in code texts.

Mediapoiesis is not just system immanent reflection. Although such works may start in that way, they are contributing to and take place in the environment of other systems. So a mediapoietic aspect is defined. For example "txtz ware" by Projekt Gnutenberg (Sebastian Lütgert, Florian Cramer, Germany, http://textz.gnutenberg.net), can be considered as a piece of software art, which reflects the specifics of a software causing reactions in the real world of legacy. The ‘programme’ of the application is a political file sharing act, where neither copyright nor intellectual property is accepted. Because of a very explicit cultural use of software a social awareness arises. This effect is also reinforced because certain software conditions are exploited and reused: a php.txt is a script published under the GNU public license. The GNU Project was launched in 1984 to develop a complete Unix-like operating system which is free software: the GNU system. GNU is a recursive acronym for ``GNU's Not Unix''; it is pronounced "guh-NEW. Therefore it can be modified and copied for free although its content is not free at all.

Mediapoiesis and the idea of code-critique

For an applied epistemology in the discursive reflection of networks and the machinic as media, that generates also a surplus, we choose a self-referential view of the relations between codes, interfaces and cultures, which touches the borders of systemics. The analysis of scientific and philosophical thinking models of 2nd order cybernetics linked to informatics and cultural theory, form the background. 9) Further the term autopoiesis has to be applied here, following Humberto Maturana’s and Francisco Varela’s initial definition of it, as practical and generative aspects are becoming evident:  AUTO means “self” and refers to the autonomy of self organising systems; and POIESIS - which shares the same Greek root as the word poetry - means ‘making’. 10) So autopoiesis stands for self-making. Consequently mediapoiesis stands for a self-making of media. The distinction between organisation and structure in this model is clear: organisation is an abstract description of relationships and the structure is constituted by the actual relations between the physical components, as the embodiment of its organisation in different surfaces of media texts. Since all changes in the system take place within a basic circularity, Maturana argued that the components that specify the circular organisation must also be produced and maintained by it. Organisationally closed but energetically open, these systems are the environment for other systems. Although these thoughts were focusing on living systems, the main idea is close to the concept of self-organisation in binary networks, observed as the spontaneous emergence of ordered patterns, with the initial state of the network as a randomly structured organism. This for example happens when network systems are introduced into the operationally closed environment of multi-player environments. Additionally a not only self organising but also continually self referring networked narrative implements a meta-discourse on the used codes and languages on both sides, those of technological programming languages and those of cultural metaphors and the epistemological conditions of these observations.

Additionally, in this concept of codepoiesis or more generally mediapoiesis, Niklas Luhmann’s argument on autopoietic theory (1984, Soziale Systeme) has to be considered, as it reflects upon organismic, psychological, and social levels of autopoiesis and therefore points towards the phenomenon of the social use of network technologies in online communities, clans and scenes.

The rewriting of grammar of the used languages proves to be further significant. Thinking of language communities, it seems helpful to take over the point of view of cultural theory. The nature and debate of the linguistic sign finds a correspondence in the standards of network protocols. When Ferdinand de Saussure speaks of the difference of langue and parole, he emphasises on the fact, that “la langue” is “the social product of the faculty of speech and a collection of necessary conventions that have been adopted by a social body to permit individuals to exercise the faculty”, 11) whereas ‘la parole’ is the execution. As both have to go together, we have to experiment on executed programmes and code lines inside the networks under consideration of the social conventions within these temporary shared communities. Secondly these conventions are then officially standardised as so called ISO (International Organization for Standardization) standards for network and transfer protocols or even for environment modelling languages like VRML (Virtual Reality Modelling Language), and they are becoming a new matter of community discussion and creation. While sticking to cultural theory we could draw parallels to the concept of a normative grammar. 12) I would like to refer here to the work of Antonio Gramsci (Normative Grammar, 1929-35), a political activist and theorist who emphasised the positive and even necessary aspect of certain forms of unity, as they may actually have an initial function to force liberation. If, as Gramsci argues, written normative grammar always presupposes a cultural tendency, and is thus always an act of national cultural politics, then also the written commercial and open standards of protocols are implicitly showing these cultural tendencies in the worldwide network and they are therefore a political act. 13)

The relationship between poetic or creative language, writing and coding, standard and normative protocols is the essential one coming up to the surface in code based networked narratives. A further linguistic model can finally be traced back to reflect these categories of environment programming languages. Roman Jakobson, another key thinker in cultural theory, refused to distinguish between different kinds of texts and brought the same mode of analysis to bear upon both everyday language and literary texts. In this structuralist theory of linguistics, functions relate to factors present in any communicative act, as there are sender, receiver, context, code, contact and message. The scheme of functions consists of emotive, referential, poetic, phatic, connotative and meta-lingual elements. The given examples and related discussion are exemplifying, that these elements form a major part of the described techno-narrativity of networked communication. 14)

published in: Andrea Zapp (ed., networked narrative environments as imaginary spaces of being, Published by Manchester Metropolitan University, Faculty of Art & Design/MIRIAD in collaboration with Foundation for Art and Creative Technology, FACT, Liverpool, 2004

[nybble-engine-toolZ home] - [A radical meta-art system shooter] -  [Narratives of the code]

1 nybble-engine-toolZ got an award of distinction in interactive art at the Prix ars electronica 2003. http://www.aec.at/en/prix/updates/article.asp?iNewsID=309&iTypeID=6, (15.07.03)

2 Margarete Jahrmann, Max Moswitzer (Austria): "nybble-engine-toolZ", www.climax.at. nybble-engine-toolZ was developed in cooperation with the V2_lab Rotterdam and a first version was installed at the exhibition "Metadata", at the Dutch Electronic Arts Festival 03 in February 2003. In the game pad statement tool installation during the DEAF exhibition at Las Palmas in the harbour connected to Showroom Mama, a gallery in the centre of Rotterdam, a number of peace messages and pings were sent out. http://deaf.v2.nl/deaf/03/archive/node/Text/show.py?nodenr=136459&domain=, (15.07.03).

3 Deleuze, Gilles, The Fold: Leibniz and the Baroque, Minneapolis, 1992.

4 Gogolin, Heiko, „Augmented Reality“, in: DE:BUG, Nr. 64, Oktober 2002. S. 33.

5 http://www.cs.unm.edu/~dlchao/flake/doom, (18.07.03)

6 While the study of computer games and game culture is still a largely undeveloped field, there is a significant body of research and critical writing emerging on the topic, in both academic and mainstream publications and

gaming websites. http://www.game-culture.com/about.html, (08.07.03).

7 “What (most) game companies have discovered is that people who buy computer games do not simply want to play those games, but they also want to use games as a platform for their own self-expression.” http://brian.carnell.com/articles/2002/07/000023.html, (15.07.03)

8 http://www.transmediale.de/03, (15.07.03)

9 von Foerster, Heinz, ed., Cybernetics of Cybernetics. Sponsored by a grant from the Point Foundation to the Biological Computer Laboratory, University of Illinois, Urbana, Illinois, 1974. A short reference to the informatics view: The programmpoiesis can be interpreted as a practical theory. It can be characterised by polycontexturality, the parallelity of one object in different contextures (transjunktion). An optional perspective in the reflection of programming languages and codes offers the idea of polycontextural logics. (Rudolf Kaehr, 1996) http://www.loveparade.net/pcl/, (15.07.03). This means that two different possible answers are valid and the Leibniz system of zeros and ones, true or not made visible on a reference foil, is no more valid.

10 Maturana, Humberto R., Biology of Cognition, 1970. Reprinted in Maturana, Humberto R. and Francisco Varela, Autopoiesis and Cognition: The Realization of the Living. Dordrecht, 1980. and Maturana, Humberto R. and Francisco J. Varela, The Tree of Knowledge. Boston and London 1988.

11 de Saussure, Ferdinand. The Nature of the Linguistic Sign (1916), in: Burke, Crowley, Girvin, The Routledge Language and Cultural Theory Reader, Manchester 2000, page 22.

12 The public source kernels of game engines, the modifcation of engines and the spatial grammar of an environment are profoundly related. "Shape grammars are typically designed for a specific application area... Several industry and academic groups have collaborated on the development of non-proprietary shape grammar standards. ...most non-proprietary shape grammars are slow to be developed and adopted and are quickly outdated. Proprietary standards, in contrast, can be revised on each new release of the application software they support." http://www.sdsc.edu/DOCT/Publications/a6-4/2_store.htm#GrammarsToday, (21.06.03)

13 Gramsci, Antonio, Normative Grammar (1925-35), in: Burke, Crowley, Girvin, The Routledge Language and Cultural Theory Reader, Manchester 2000, page 281.

14 Jacobson, R, Language in Literature, ed. K. Pomorska and S. Rudy, Cambridge, 1987.