November 2004 Archives

due to my general resistance of offering much (any) working code to CMP i'm being asked 'where's the computer science?'. i have a solid reason for this, most of the work i'm reading that's claims to had an arts edge falls over when you try hold it up against arts practice and theory. indeed often work presented at a Phd or Msc level contains concepts of art that would fail at undergraduate level arts studies.

there are many good and interesting tools out there for human movement simulation, well simulating body mechanics, but they are mostly put in the hands of people who don't really know how to use them and developed by people with a minimal understanding of movement practice. i'm not claiming that i'm going to come up with a total solution but i would like to help create a paradigm shift.

a blurry line between animation and simulation only clouds the issue further. i however, argue that if the goal is to enable interactivity, then simulation is the only way forward. yet animation perspectives have a stranglehold on simulation work, this leads to the odd assertion that human motion is distal in nature (ik) where as it is usually proximal (fk).

for example in the current paradigm a punch is guided by the fist, with the shoulder and elbow etc following paramaterised constraints. in a dynamic environment this will create a 'weak' punch. Punches are driven from the back and shoulder (depending on technique) with a specified alignment driving the elbow to fist chain. as humans we use a form of ik solving to target the punch but it is driven by fk. consider the fact that when we use stick based objects (bats, poles, pens etc) the far end of the object is under our control by manipulation of the near end.

there reason for this inversion is simple, it's the simplest way to animate existing character rigs. i've already noted here that most rigs (hierarchical skeletons) are under specified for not only dance but general human motion (simulation not animation). my position is not only that more advanced rigs are needed but also a different method of driving' such rigs. this is where a comprehensive method of computer based notation comes in. but developing the notation is not enough

danceforms (credo interactive) is a good example here, it's often critisised, in the past by myself too, but it's not the software at fault when you see those naff dance animations. it's the animator / choreographers. i digress. my point is that Rhonda Ryman's Ballet Moves is the bench mark for what should be supplied with dance animation software, a well developed corpus of movement notation.

whilst modern and contemporary dance is more challenging to create with danceforms (i'm exculding merce cunningham's work for obvious reasons) ballet moves allows the ballet choreorgapherto make work, rather than animate, for example

• 'Profokiev Movements for Piano and Orchestra' by Jimmy Gamonet
• DanceForms 1.0 [9mb or 45mb mov]
• Stage production [1.5mb mov]

unexpected? it really shows what can be done with good software and related resources. whatever system i develop i intend to make it simple to create notation libraries. then it would be feasable to encode many dance techniques and have a swiss army knife of a choreography tool.

politics and and tap-dancing all in one 'Tap Dancing into Syndication and Podcasting';

One of the people to ride the Podcasting bandwagon to fame - at least in the small world of people who care about Podcasting - is tap dancer Sondra Lowell, a news reporter who always manages to infuse her news stories with a strong dose of entertainment. In her press release, she describes herself as the "world's first tap dancing broadcaster."

have a listen [606kb mp3] it really made my morning, who said old time music hall was out of touch.

via: Dance and Technology Weblog

update: it seems that this may be a fake:

Steve Garfield emailed me yesterday after my post about Sondra Lowell's tap dancing podcast to share the disappointing news that the tap dancing that you hear in the background of the news piece is not really tap dancing at all... more

some notes that may shed some light on a prior post. i'm working for a computer based notation that embeds the knowledge about a particular movement as well as the geometry. most computer based movemnt notations are inarticulate, i hope to go some way towards resolving that, anyway:

Constellations are groups of stars (body locations) and consist of three stages

• rendezvous: formation of constellation
• procession: transition of stars within the constellation
• destination: final alignment of stars within the constellation
• constellations are mapped using their own frame of reference and consist of:
• node star: sacrum ('fixed' frame of reference)
• base star: center for floating frame of reference (fixed when defining constellation).
• reference star: parametrised alignment to base star.
• system stars: the remaining stars of in a constellation.

when a constellation is 'dialled in' it can occur at any location allowed by the system (class) rules. once the location has been decided the next departure point can be calculated as we already know the structure of the constellation. Furthermore each constellation contains a list of semantically related constellations (and the potential locations of such constellations) to the order of two (departures).
Clusters are groups of 'reference' stars, located around the node star. where as constellations describe fine motion clusters are used to describe gross , multi limb movement. Like constellations, clusters have three stages; rendezvous, procession and destination. The body is always in a cluster containing constellations. Clusters without defined constellations revert to default constellations. there are four types of cluster:

• Super: the default cluster of a technique, also defines the basic parameters and rule of the technique.
• cluster (normal): group of reference stars.
• globular: clusters with 'loose' parameters (structured improvisation)
• open: clusters with without parameters (free improvisation, epikinetics)

note:

1. constellations may also be 'globular' or 'open'
2. constellations and clusters may be re-navigated
3. constellations may be self governing (hand holding object, foot whilst walking etc)
4. clusters may be self governing (balance)

clusters and constellations can be used to enable interaction with other galaxies and objects in the universe, by alignment and intersection. this should allow directing, yielding and adhering, ie:

• chi sao
• tu'i shou
• contact improvisation
• pas de deux

both perception and recognition of star groupings are important, each offers distinct functional and artistic options.
the use of the term 'constellation' is a significant modification of the HamNoSys 'hand constellation' principle and realated work by Dr Richard Kennaway.

for a weekends work i would rather have the physical movement, but still, this is at least something. i'm not going to bore you with a detailed explanation, when i have it worked out in more detail i will give a dance example. this is not really for human use, but computer implementation. the layout is to aid human editing, and development visualisation (it's still a rough version). on top of this would lie a more human friendly notation or interface when needed. anyway ...

gesture
{
t0 = {b(im k q): c1(im p q): c2(im k q): c3(im k q);}
t1 = {c2(im k q);}
t2 = {c1(im k q): c2(im k q);}
t3 = {b(im k q);}
t4 = {c3(im k q);}
}

t - time
b - absolute node
c - relative node
i - interpolation
m - interpolation parameter
k - kinespheric vector (xyz position)
q - quaternion (kinespheric orientation)

can't wait to start some of my studio based observations and be around dancers again..

note: technically the relative nodes use relative coordinates rather than an absolute, kinespheric coordinate. i've used the k in the notation because the kinesphere is not the global frame,the k reminds me of that. the final version I will have a more accurate notation and also show the global (stage) coordinate of the kinesphere frame

as you can imagine research students at cmp are given a computer for personal use. normally this is a desktop, but i asked for a laptop due to the fact that i would have to be 'out and about' as part of my work. well today this has really paid off, and i'm thinking that i would like to work like this more often sometimes.

the set up:

• studio
• dancer
• laptop
• digital camera
• digital projector

so i'm in the studio with the digital camera attached to the laptop shooting at 2fps and the projector facing 180 degrees from the camera. this allows me to dance and see the images being taken at the same time (i would prefer not to be the subject but there are too few dancers on campus).

beginning with an improvised phrase i refine the movement until such time as i am happy with the results, i then notate this in a rough version of the implementation code i am working on. i then compare my written structure with the the images via a fixed reference frame that i overly each picture with. with slow(ish) movement you can get a reasonably realistic playback when using a sideshow mode.

now i could use video, or motion capture to do this kind of analysis, but when it needs to be 'quick and dirty' still images are best for me. what i was looking at today was the facility to manipulate the COM of the bones we use during movement. i'm cheating slightly by taking the midpoint of the bone as the COM (for simplicity) but it's not too important for my purposes. what i want to look at is if we can:

1. find end positions using bone COM
2. navigate space articulatly using bone COM
3. touch using bone ends (ie finger tips) via realtive placement of bone COM
4. discribe the position and orientation of a bones COM verbally and physically as a teaching / performance tool

these are early days but it's helping me to solve some implementation issues that i would otherwise have no point of reference for. over the next few months i intend to visit a range classes in differing locations and observe, document and analyse people learning dance techniques and dance works. if all goes well this should start off with Steve Paxton's Material for the Spine' workshop at independent dance at the end of this month.

from there on i'm hoping to go to various dance colleges and independent workshops etc in the UK and Eire. i should get back to what i was doing now... this room has a clear line of sight to a WiFi antenna, Internet in the studio is useful but distracting...

in sci-art projects there is often talk of developing a shared language in which work can be discussed. such a concept has been central to my work at UEA due to the nature of my studies (computer science). however rather than developing a shared language i believe that a shared ideology and conceptual framework is more important. language is bound up in the epistemology of a specific domain, thus sharing language requires a high standard of multi domain knowledge. such knowledge is not often available.

furthermore there is rarely and equal sharing, usually one domain is required to conform to the other for the purposes of implementation or where a lack of rigorous practice is perceived. here i am learning how to present dance through the language of the computer science. as dancers we draw on a range of concrete knowledge that we have little descriptive access to, our practice abstracts the very tools we sometimes need to describe our work.

Physics, alignment, orientation, interpolation, angular and linear velocity, angular and linear acceleration, center of mass, duration, distance, force etc etc are all essential to developing artistic practice. yet, in such practice an implicit understanding is of greater importance that descriptive, or calculated proofs.

so this is how i often have to talk now, about the many faceted theoretical aspects of physical movement. this and developing ways to express aspects of creativity through similar means. yet there is a big gap, from the other direction in understanding the application of such theories. dance epistemology (theory and practice) are rarely examined in the detail that is deserved by those seeking to simulate dance. more over there is a general lack of understanding of the performing, teaching and creative processes.

i this context i am being challenged to prove my practiced based movement simulation metaphor through implementation in a digital environment. my question is this; do existing concepts of movement simulation hold up when applied to physical practice.

i think you already know what the answer is...

'Movement is a displacement or change of position, even if it cannot be defined as such' - Merleau-Ponty
motion is central to dance, be it sensed, perceived or implied.

this is the issue central to human movement simulation (HMS), at least it should be. with few exceptions HMS research has paid more attention to function (biomechanics) and posture.

the prevailing concept is simple, simulate the way in which the human body works (inverse and forward kinematics, inverse and forward dynamics with a hierarchal skeleton) and motion will be generated as the product of changing posture. this is key framing in three dimensions, where the transition between postures; interpolation is dependent on a single solution (such as quaternions) or the biomechanic construct (kinematics, dynamics).

the problem is that motion generated according to these principles is rarely naturalistic or believable, why else would motion capture be so prevalent. key framing works well in animation because the visual effect of motion is easy to simulate in two dimensions as numerous optical illusions demonstrate. whilst the visual effect of motion can be simulated in three dimensions it still requires accurate physical and transitional modeling.

remember how at the end of a hard ballet class you might start cheating, not faithfully applying ballet technique but making sure it appears that you are. that's simulating the visual effect of motion in three dimensions. an experienced eye (such as your teacher) can tell the difference but otherwise, there is little visual discrepancy.

this is what we do as dancers when learning new movement and techniques, we simulate the visual effect of movement whilst applying our knowledge of somatic function, potential and existing movement skills. it's the application of this detailed knowledge that is missing from current HMS solutions.

the knowledge relating to specific areas, or domains of movement practice are not only detailed but difficult to encode. solutions such as style based kinematics and dynamics have been developed from parameterized data sets (created from motion capture of the specific domain) yet these also suffer from simulating the visual effect rather than applying the physical technique.

animating, performing and teaching movement are very different skills. with HMS the system, or golem (avatar) must be taught how to dance. yet a this teaching must be encoded in manner that can be implemented. This encoding requires a knowledge of movement simulation approaches and solutions.

a common mistake in encoding domain specific knowledge for HMS is taking the 'beginners' approach. the learning curve of a golem is very high, the key is in encoding the application of the technique at the same time as the function. the beginners approach can be seen in solutions where domain specific postures are encoded with little attention to the domain specific transitions (application).

clearly encoding domain specific knowledge requires a high level knowledge in a) the specific domain being encoded and b) the methods by which such data can be encoded and the target system. in most instances this cannot be achieved by a single person, but by two generalists with complementary skills.

of course to fully, and faithfully encode a movement technique requires the golem to have a body capable of said technique. few avatars a modeled in enough structural detail to achieve realistic walking let alone advanced movement practices. areas of the body that require particular attention are the spine, pelvis, sacrum, ankles, feet and toes. i'm not going to explore this in detail here but it's an important issue.

so how is the knowledge encoded so it can be reused? through layered notation. this not only speeds up the encoding process but also enables more 'novice' users to generate motion sequences (it should be remembered that crafting motion is still a skill, some 'technical' knowledge of the simulated domain is still required).

there should essentially be three types of notation that may form multiple layers; raw, abstract and compound abstract. the raw notation should be able to describe all motion. typically the raw notation will be math based with structural references, it need not be entirely human readable / editable but should be formed in such a way that a simple interface may be used create and modify values.

abstract notation is formed from raw notation. this direct abstraction allows for the quick reuse of movement concepts such as the plié in ballet or the fist handshape in sign language. such concepts should not be a part of the raw notation as they are domain specific constructs that would limit it's descriptive and adaptive flexibility. whilst it is tempting to consider them basic movements they are actually compound structures.

compound abstractions may combine raw and abstract notation, multiple abstractions, abstractions and compound abstractions or multiple compounds. raw notation may used to generate specific modifiers. a set port de bras would be defined in compound notation.

the advantage of this layered structure is that the detail in each level of abstraction is maintained allowing discrete changes to be retrospectively applied. this also helps to define stylistic constraints within a domain, therefore ballet can have styles such as:

• Balanchine
• Bournonville
• Checetti
• Vagonova

these may be defined after the construction of the basic domain, with specific notations for each style. the same is true for other movement practices such as T'ai Chi Ch'uan, styles include:

• Ch'en (陳氏)
• Yang (楊家)
• Wu Hao (武家)
• Wu (吳家)
• Sun (孫家)
• Wu Tang (武當)

conventions could be specified as domain:style:abstraction. the need to define style within the notation highlights how descriptive the raw notation must be, style can be identified by; posture, orientation, alignment, transition and application. these parameters should not only be defined in the raw notation, but be implemented in the simulation engine.

the notion of 'avatar independent notation' is currently very popular. Taking the position that golems (avatars) a time consuming to produce, a notation should be designed for use with any avatar. this is not possible to two reasons:

in specifying a notation and encoding motion you define the requirements of the avatar. for example, if you encode sign language your avatar must have hands, fingers, elbows etc capable of performing the movement all movement practices are based on a default set of techniques that are adapted to individual capabilities, and application. alignment within transitions and postures are variable within the parameters of the technique or visual effect required.

thirdly, (as i commented earlier) not all golems have the correct structure to apply movement techniques correctly. for example 'turn out' requires a sacrum articulating with each half of the pelvis. in most avatars this three segment mechanism is fused into a single segment. you may have observed that digital ballet dancers that don't use motion capture have poor degage, this is partly due to single segment pelvic girdles.

i believe that notations should be designed for a default avatar, this allows the basic encoding to be adapted for other avatars (semi) automatically. when re-mounting a dance work you look for dancers that can fulfill the needs of the choreography and adapt the work to them where required. this is the model i suggest for the digital space. the this forced adaptation of the notation would also result in a 'individulal' style for each golem. in multi golem simulations this would increase the realism of the scene.

the HMS solution i am working on is based on my dance practice, particularly my improvisation practice (including CI). the detailed study of the fundamentals of human movement and the cognition of that movement directly apply to simulation design. i may expand on specifics later, but for now...

thank's for reading

i've been neglecting my practice, with all this reading, problem solving and abstracted analysis there has been little time for motion. this is partly due to the lack of space, having no ready access to a studio is restrictive. when i was studying for my MA i would practice after the teaching day had finished for a couple of hours each day. it was a real time of growth... and what in part, lead me here. the importance of regaining that kind of time cannot be underestimated.

this is not an issue of fighting for the arts within my scientific context. nor feeling 'out of touch' with my artistic roots but a recognition that the knowledge embedded within my practice is more keenly recalled and transmitted when refreshed daily. sci - art practice is trendy, i'm not trying to be fashionable, but do what i think is important. this isn't dance (technology) wrapped in a veneer of (computer) science to make it credible, but to show what can be achieved through a combination of both disciplines.

that said, i don't think dance is always able to articulate the scientific principles that underly our practice well enough or often enough. beyond art for arts sake (something i do believe in, as well as 'high' art) there are several sold principles that can be used to back up our practice in varying circumstances. such scientific knowledge or qualification does not lessen the artistry of what we do, but sits alongside, mostly silent, until we engage in activities that requires it's application.

as my interest is in the simulation and synthesis, rather than the animation of dance the scientific concepts are speak louder than usual. what is implicit motive knowledge to the physical dancer must be 'taught' to the avatar (or golem, as i like to call them). this teaching does however involve aspects of artistry, making the golem appear realistic in say a ballet sequence, requires transmitting (encoding) the principles of ballet technique and performance. to attain something of the 'x' factor in the performance (in addition to the standard principles) we might use, motion texture (perlin noise, EMOTE etc.) or complex / emergent systems.

however we apply 'x' factor, it is worth remembering that not all dancers have it in the first place. therefore adding such 'textures' to a simulation is not covering up an inherent fault in the modeling but an accurate representation of real world circumstance. development, learning, education, training and communication are the face of many dance technology environments (and tools). computational aesthetics and interpretation deserves more attention than it currently receives.

interpretation is not segmentation and recognition, digital dancers require more 'sensitive' skills. 'reading' movement and providing an 'appropiate' response is a complex task. whilst mirroring, inversion, call & response and sectional mapping may be valid responses, any system limited to these options will quickly reveal it's limitations. to generate aesthetic responses we need a comprehensive way in which to generate movement that relates to the way in which movement is 'perceived' and notated.

this also applies to interpreting graphical dance notation where we need to transform rather than translate the notation. mapping from two to three dimension and generating motion from static marks.

there are a wide range skills and processes to model. teaching the golem a set dance or dance pattern is not enough, the golem must be capable of creating it's own:

• transitions (movement)
• positions (to define location, limbs and allow 'reading', positions can be active of motive)
• transcription (facilitating abstracted notation, interpretation & translation, inter{react}activity)

these capabilities should exist in two and three dimensional constructs. from informal choreographic scribbles to live motion capture data my goal is to retain 'concept, context, content' in movement (dance) simulation. as Elizabeth MacKinnon [*] comments in her blog motive (reason to move):

These artists are working with technology because it makes sense, because it's instrumental, and even maybe partly the subject matter, but never letting the technology simply drive the product without any attention to concept, context or content. You don't have to ask what the point is. [concept, context, content] she is not talking about me, and i'm sure many people question the 'point' of my work, asking why i want to replace dancers. well i don't. in simulating the dancer i hope to reveal aspects of our practice that would otherwise remain hidden due to motive bias. i'm not looking at emotive, but motive responses to a range of stimuli that are cognisant, yet interpretive. concept context and context are at the heart of my work.

matt rss 2.0

[*] Elizabeth MacKinnon of collective (gulp) dance projects is currently studying for her MA in digital performance at Doncaster College. her blog is one of the few dance related links on my blogroll and i wish her every success with her practice and MA studies.

from sheila, via communitydance.org

The Chair of National Youth Dance Trust, Susannah Simons, today announces that following a review of its operations trading under the name National Youth Dance Company, National Youth Dance Trust, with the support of Arts Council England, is to link up with Youth Dance England, the newly formed national organisation for youth dance established through funding from the Department of Education and Skills' Music and Dance Scheme. as a former member of National Youth Dance Company I think this is rather good news, i've often advocated that NYDC should be the benchmark for youth dance practice, and that the regional companies should scale to it's level, so perhaps something like that may start to happen.

Anne Went (Middlesex University) also deserves a mention for her innovative and inspiring dance training, I learned a lot from my time with NYDC and I hope it will benefit many other young dancers in the years to come.