Klaus visit's at JPL this Summer

Professor Klaus McDonald-Maier, who leads this collaborative EPSRC Global Engagement Project with NASA, visited JPL this Summer. Klaus is interested in looking at increased reliability for hardware and software architectures in robotic systems. These systems include extra-terrestrial robotic rovers such as the Curiosity Mars Science Laboratory, which recently successfully landed on Mars where it will undertake a series of experiments. Curiosity relies on complex programmable control systems and an enormous amount of dedicated software which must be extremely reliable as it is deployed in such a remote location.

Klaus commented ‘It was extremely exciting being at NASA JPL when Curiosity successfully landed and it will be very interesting as it embarks on fundamental experiments which should give us great insights into the structure and characteristics of Mars.’

Maria visit's to JPL this Summer

Prof Maria Fasli, co-investigator in the project, visited Adrian Stoica at JPL between the end of July and early August this Summer. Maria is an expert in multi-agent systems and she is looking at mixed-initiative systems for robot control with Ardian's group.

Collaborative Brain Computer Interface for Simulated Spacecraft Control

While at NASA JPL in the Summer 2012 working with Adrian Stoica, Riccardo, Caterina and Francisco experimented with the real-time control of a simulated spacecraft using the brain signals from two users.

Here is the setup

The subject on the left is Riccardo. The subject on the right is Luis Clark, a student volunteer.

Here is a close-up of the simulator

The objective was to pass as close as possible to the planet (intermittently visible in white). Here is a movie of one of the simulations

The development and experimentation of the software had started some months before at Essex and are still ongoing. Preliminary results on this line of research will appear in the Intelligent User Interfaces conference in March 2013 in a paper entitled "Towards Cooperative Brain-Computer Interfaces for Space Navigation". 

Here is the abstract:

We explored the possibility of controlling a spacecraft simulator using an analogue Brain-Computer Interface (BCI) for  2-D pointer control. This is a difficult task, for which  no previous attempt has been reported in the literature. Our system  relies on an active display which produces event-related potentials (ERPs)  in the user's brain. These are analysed in real-time to produce  control vectors for the user interface.  In tests, users of the simulator were told to pass as close as  possible to the Sun.   Performance was very promising, on average users managing to satisfy the simulation success criterion in 67.5% of the runs.  Furthermore, to study the  potential of a collaborative approach to spacecraft navigation, we  developed  BCIs   where the system is controlled via the integration of the ERPs of two users. Performance   analysis indicates that collaborative BCIs  produce trajectories  that are statistically significantly superior to those obtained by single users. 

Do acupuncture points provide better or worse electrode locations for BCI?

One of the studies we did at JPL during the Summer was to investigate the effect of supposed acupuncture nodes on scalp electrode/skin impedance (which may result in unusual EEG readings). We could only do a limited number of measurements due to the limitations of the equipment we had with us.

Lois Clark (a student) volunteered to be the subject of the tests. Below is the motage we were able to put together. It places 6 electrodes approximately 1cm apart in the area of a well-known acupuncture point.

We found nothing significant (albeit with one participant only), but a larger study will be run in the future before the hypothesis can be ruled out.

Some of our researchers visiting JPL

Here is a picture of some of our team members visiting NASA's Jet Propulsion Laboratory in July 2012, just when the Curiosity was approaching Mars. For the occasion JPL set up a display of copies (some in scale, some 1:1) of various rovers which over the years have been launched by NASA.

From right to left: Riccardo Poli, Caterina Cinel, and Francisco Sepulveda (all from Essex), Adrian Stoica (our main collaborator at JPL) and Luis Clark (a student who acted a few times as a subject for our experiments at JPL).

Riccardo and Francisco's visit to JPL

Francisco and Riccardo visited Adrian Stoica's lab at JLP in late May 2012 interacting particularly with Adrian but also Christopher Assad (he is specifically interested in EMG applications to robotic control and prosthesis) and other people.

They were also invited to give a tutorial on BCI to get people in the lab up to speed on the state-of-the-art and recent developments in the field.

The seminar was entitled "Brain Signals + Evolutionary Computation = Human Competitive Brain Computer Interfaces". Here is a summary:

The keyboard and mouse provide us with reliable, but unnatural forms of input, being primitive transducer of muscular movement. People who lack muscle control cannot used them.  Wouldn't it be nice some day to be able to replace the mouse and keyboard with systems capable of directly interpreting the intentions of computer users from their
brain activity? 

This is the goal of the field of Brain-Computer Interfaces (BCI). Unfortunately, this goal is hampered by a number of problems: brain signals are typically extremely noisy, they vary in location and temporal dynamics from subject to subject, they depend on the age,
tiredness, attention, food and drug intake of subjects, etc. So, even the best BCIs are extremely slow and prone to misinterpret user intentions.

In this seminar we will briefly review the different approaches to BCI, with particular attention to non-invasive EEG-based BCIs, highlighting their difficulties and limitations. We will then illustrate a number of cases from our own research in the Essex BCI group, where evolutionary algorithms and genetic programming in particular have helped develop systems which are competitive with human-designed ones, thereby accelerating the development of practical BCI technology.

Potential Simulators for the Project

Caterina and I (Riccardo) have done a search of space flight simulators, particularly focusing on anything written in Python or that can be interfaced to Python (since this is the language we have used for our BCI Analogue Mouse system).

Here is a preliminary list:

• JPL have developed and used in a number of missions their own simulators. Two have caught my eye:
• DARTS (which stands for Dynamics Algorithms for Real-Time Simulation) a general simulator for multi body dynamics (and on which other simulators are based). This could be very good for simulating the flying of a spacecraft.
• ROAMS (which stands for Rover Analysis, Modeling and Simulation) which would be good for simulating rovers performing missions on a planet or something.
• Vega Strike is a sort of a game of space exploration and missions. I've downloaded and installed version 0.5.1 on Ubuntu and it worked really well (note: I did not get it from the ubuntu repositories, since that is partly broken). Apparently one can create missions by Python scripting, but I haven't got round to test that. It would give us the possibility of running really realistic simulations. (One issue to consider is whether our BCI Mouse software would interact ok with it, since it grabs the mouse pointer and goes full screen.)
• Space Commander is a physically realistic simulator of space travel entirely written in Python. I've already committed and adapted version 0.4 of the simulator (see changesets r3 and r4). Because it is written in Python and it is very compact I would suggest to use it to do the first few experiments. (Incidentally, it uses pygame, which I think is the same package Psychopy uses: Psychopy is used for stimulus presentation etc. in our BCI Mouse.)