Not really vis but this was important in the day.

Back in the Computer Lab., at the University of Cambridge in about 1992 there were at least two filter coffee systems; both running on a slow drip-filter setup. The Rainbow Group had access to one machine in a walk down a corridor to a small kitchenette; but the network team had to travel a lot longer way to their coffee machine. A problem of lack of enthusiasm to travel meant that often they would find an empty coffee pot – or even worse stale coffee. A solution was a camera video stream that linked to the linux server and stream images to X Windows. So when new fresh coffee arrived this would be obvious from the image on everyone’s desktop.

http://64bits.co.uk/ have been rebuilding old systems and in East London they rebuild the Cambridge Coffee pot scenario – with similar equipment. There is a wikipedia page on the original: https://en.wikipedia.org/wiki/Talk%3ATrojan_Room_coffee_pot

Is video streaming – visualisation – well definitely yes as it is an interpretation. Due to low quality network speed and CPUs; the small 128×128 video stream of images had to just include the coffee level and also the fps was about one frame every few seconds – but that is a good rate and good enough resolution for “coffee pot observation”.

Touchscreens are all the fashion – but  any system you deploy is expected to work smoothly and intuitively – including visualisation ones. There are some company purpose-built systems for exhibitions that are far from cheap although they may work well they can also suffer from lack of agility when recreate new content. There are also simple tablet/phone software options available for touching and manipulating volumes but can have limited graphics ability.

Over the last couple of years we have collaborated informally with partners to share knowledge on open source solutions for large scale volume visualisation (say greater than 256 x 256 x 256) on a touchscreen device.

A hardware specification service has evolved where we can assist with advice and quotes for touchscreen and workstation visualisation nodes, as well as more importantly related software installation; used for exhibitions and PE. A wiki is available at:

http://tyne.dl.ac.uk/twiki/bin/view/Visualisation/ProjectsTouchTable

At Manchester, connecting through Research IT, there are now groups available to informally share equipment and expertise – and as importantly share experience in interacting with the public and other scientists – even if their research data content is often diversely different.

• MRI medical data e.g. exploring white-matter changes within the human brain: Geoff J M Parker, Hamied Haroom, Saray Parkes (ISBE)
• Multiple Materials Science analysis e.g. cracks, fibres and mechanical defects: Sarah-Jane Clelland, Phil Withers (MXIF)
• Fossil and everything ancient within the Manchester Museum: Russell Garwood, Alan Brown, Campbell Price, Roy Wogelius (ICAL)

Both being asked to be available for the upcoming Presidents Office organised ‘University Celebrations of Regius Professorship’ on 25 April 2017 in the NGI.

I was having to wait for a train at Euston Station in London- and visited the Welcome Trust that is opposite, and they had an exhibition on the story of electricity.

One neat bit of 3D visualisation was near the end of the exhibition and showed a 1950s Electrical Consumption Graph:

“This 3D graph, compiled by the planners of the Central Electricity Generating Board, represents the daily electrical energy consumed over a period of two years during the 1950s.”

Unfortunately, not very interactive – with the glass protection, although you can view it easily from many angles; but a few things were right in that they chose two years of data. This is a trick used today even when there is only one year’s data available – where you recommend copying the data – and was used by Florence Nightingale in her more radical circular statistical plots (of morbidity rates during the Crimean war). The problem comes with matching the ends up – between January with December in most calendar statistics. These should be continuous, but the human eye has difficulty in spotting smoothness etc. If you can not use circular plots and in this case with card indexes this would have been very difficult, then by repeating the data or here using two years of data, you can see the changes between December and January as clearly as  any other pair of neighbouring months.

In principle the whole medical topic of radiotherapy analysis is full of visualisation. A recent sandpit discussed Monte Carlo techniques that are critical within the radiotherapy planning / beamline construction / or just radiation safety / etc…

So where is the visualisation interests as the main need is to make these processes faster and more useful. We did pitch three small projects to look at:

1. Solving the 1:1:1 problem may involve visualising the results immediately after compute and not storing or transmitting this.
   The 1:1:1 problem termed by radiotherapists is to model 1mm cubed, solved in 1 second with 1% accuracy for practical use.
   There will be a one-day workshop where the required (back of the envelope) 250x speed up will be discussed with UK researchers and industrial participants; from the software / hardware / and remote access skills.
2. Sharing data discussed ways to share both code and data: an example of data / code share that worked is on zenodo (yes have used github as well).
   – Data set at: https://zenodo.org/record/16474#.WMlng_ni7Lk
   – Code for reconstruction at: https://zenodo.org/record/16539#.WMlnlvni7Lk
   – Even has a published tutorial: http://eprints.ma.man.ac.uk/2290/01/covered/MIMS_ep2015_26.pdfThese have official DOI tags and therefore are version controlled for both code and data so then can be used in papers by others – and have been,
   http://eprints.ma.man.ac.uk/2377/01/tosca_scoban.pdf
   http://www.imm.dtu.dk/~pcha/HDtomo/JCLW16.pdf
3. A final project is looking at fractal analysis from CT to MRI analysis and considering the problem from 3D to 2D to 1D. This will enable new planning processed to occur.

More details to come. This was organised by the Advanced Radiotherapy network https://www.advanced-radiotherapy.ac.uk/ and thank them for allowing these ideas to be pitched.

There were global other issues and “Focus Where it Matters”, was a industry quote and will reuse here; so they need not just to increase speed but also to control inaccuracy and inconsistency.

Had the great privileged to be one of the judges at the Manchester Grammar School, Science Fair Projects involving about 200 very enthusiastic future scientists. Each group, of three, had to design and evaluate a scientific hypothesis applied to sport, which was the theme for this year; e.g. do balls roll faster with change in temperature, does sugar consumption relate to concentration performance analysis, …

All did very well, but there was a good debate and points raised on how to present their work results- should they use bar charts, line graphs,and what scale ranges are important and if error outliers should be included. Interesting discussion also occurred on the quality of the graphs, produced automatically by computer software programs and how to change these.

It was good at this age that they could grasp some of the concepts that seem illusive to certain older more mature researchers when they present their work. A key lesson learnt was not to spend all your effort creating wonderful results, if the presentation and visualisation story is not given a proportional amount of time to be created.

Visited EON Reality – UK headquarters are just up the road from Manchester city centre – very convenient as only a couple of tram stops away now.

They have always been involved with various CAVE technologies; including a fun training system shown on their portable (dismantles into two crates) system, 

One feature would like to consider is how their 3D software works with volume and engineering visualisation – and as discussed with the guys – how the VR headset systems produce as good an experience (if not better). An important part was the ability for the sw to display on all types of platform from tablet to large scale VR system.

EON Reality had a whole set of objects including standard curved walls and  a reverse curved perspex type screen for exhibition type spaces. To complement this there was a training academy for 30+ students each year who could understand not just the sw but the whole production life-cycle.

Thanks to Ros at Digilab http://www.library.manchester.ac.uk/using-the-library/students/initiatives/digilab/ for inviting us over and showing some of her new interactive teaching and research equipment. Couple of VR/AR items and a discussion on how they would solve understanding scale and context issues for visualisation of large volume data sets.

HoloLens One of the early UK released versions of Miicrosoft HoloLens (top right image) was available and impressed due to its response and quality. The demonstrator was a game allowing you to blast holes in the walls of the room, where large robot insects would emerge and then attack you. Is a very immersive experience of Augmented Reality and worked extremely well with rapid head movements as well as correctly obscuring the right parts of the real world – so was absorbing.  Discussed science based planning exercises where;

• Items on the floor – could be marked up as discovered; geology pointcloud mark-ups.
• Volumes could co-locate with real objects and be cropped/clipped on demand (“shot at” metaphor)
• …

One key failure in this version was the small field of view – as had to move your head often and setting up the glasses on your head was an issue as took a few attempts to see most of this small FOV. This in future versions will improve – just needs more compute power and resolution!

ROVR An add on movement system (top left image) – relatively cheap one – was shown by http://www.wizdish.com/ the ROVR. A simple idea so while you are standing in a slippery bowl with slippery (low friction) shoes you can use standard VR headset and slide your feet (like walking on skies without lifting your feet). This feeds back via USB forward speed and rotational direction for very intuitive movement. Played a immersive pacman running game and again experience was great – not needing the main protection bars once sussed out balance. Highly impressed as allows:

• You can travel distances and appreciate the distance; say along a feature in a visualisation.
• Comprehend scale and distance – for example travel to a location and see scale of fature in 3D volume set – eg cavity.
• …

Will be tricky to move from first or third person mode for viewing  as not directly connected to the framework for context but worth watching and programming for.

This was a wonderful little term mentioned in passing at a synchrotron vis meeting. It sort of means what can be thrown away and still produce the message or story that the visualisation wishes to convey.

There are two areas where data can be thrown away – from the original or derived data set so you select parts that are appropriate; or from the items you wish to show in the visualisation itself – cropping isosurfaces or streamlines for example.

A tomography pipeline operation should be mentioned that addresses the 100GB problem – and could be said to be the first half.

100GB Problem

I have a scanned 3D data set that is about 4k x 4k x 4k in size, with 16 bits per voxel grey scale we have, 128 GB or raw data. How do we visualise this.

We can just use lots of CPUs and GPUs and this is fine – although not necessarily straightforward. See video from (TO Upload)

Do a simple dataflow so steps:

• Load the complete data set into a fat memory workstation – you have to find one of these but there are ‘many’ 1/2 TB RAM systems out there.
• Volume visualise the complete data set that works on simple GPU parallel code.
• Select volume of interest
• Crop this volume – aiming for about 1-2 GB
• Extract this sub-volume and then possibly scale to 8 bits per volxel
• You have a data volume about 1/2 – 1 GB that can go into your laptop for normal visualisation and hand editing / markup. Simple.

Not always practical but then there are lots of cool code that only works on <2GB volume due to meshing , level-set analysis and your heart and CPU are freed.

Important to go back to the raw volume and check you are have the right conclusions.

Imaging and visualisation has at its heart coding so the emergence of the Research Software Engineer as a career by being supported by the university, industrial and Research Council sectors is very welcome.

Vis From Manchester Central to being On the Top of The World – Still making the user via visualisation be deep within the HPC-Loop

Had the opportunity to present ideas of integrating human visualisation within the HPC (high-performance computation) loop. On 14 December 2016, Computing Insight UK 2016 launched with over 250 delegates and suppliers present; it was a great session to review the use of Visualisation within the Hartree Centre and describe how it has been important to keep the human in the visualisation/computational loop. This included the use of multi-use vis and discussion spaces as well as incorporating fat-memory GPU nodes at strategic locations; and then defined a future proposal to have an infinite job submission system that would stop only under human-visualisation control.

Needs for RSE to be integrated and employed

This talk was then modified and semi-repeated for a synchrotron (x-ray imaging) related workshop, an EU COST / PSI event on 9-12 January 2017 in Switzerland, which focused on the visualisation of complex imaging for a specific audience. For this we need software developers who can look at specific problems for users and have the dedicated time to create these solutions.

We submitting an EPSRC proposal with Manchester Research IT Services, for X-ray Tomographic Imaging, which is soon to successfully launch a new Flagship grant program in April 2017 – this will fund two RSEs from Manchester; Daniil Kazantsev (included in photo above 5th from right) and Jakob Sauer Jørgensen to revive the reconstruction codes within the reconstruction library; specifically these are for complex multi-channel data. Starting for three years both will be employed by the University of Manchester, but Daniil will be permanently based at the RAL – Harwell Campus next to the Diamond Light Source.

Links:

• http://www.ccpi.ac.uk
• http://www.stfc.ac.uk/ciuk
• http://www.itservices.manchester.ac.uk/research/

Was at a show session in London – hosted by jigsaw24 – with a range of the new headset VR  (all except Sony [IMO one of the easiest and most comfortable to wear – possibly because it is well balanced]) creators and some hardware suppliers. 100+ attendees, with a mix of commercial and academic interest.

After a techy roadmap

• EditShare (showing mainly 360VR data capture and pipeline use) emphasising on data storage issues and;
• HTC Vive with the new Nvidia cards for automatic optic distortion was good – as always nvidia themselves create great prototypes (see their circus games in VR – with PhysX and some tactile feedback);

An interesting sales pitch was as a full system can use up to 4mx3m in space with multiple tracking nodes – there is an opportunity for nvidia to sell a new high-end pc in a new location at home. This space is too big for many living rooms; but the garage is an ideal (when empty) location to host a VR studio to achieve the best experience – so you will need a new PC (with high-end Pascal architecture quadro card!) for the garage.

Couple of interesting add-ons were shown but not demonstrated live from HTC – the new wireless modes which add only a few ms of latency, but allow you to wear a headset and do back-flips and other gymnastic maneuvers, a common occurrence in the future! The other was the HTC Vive Tracker a small attachment you can fix to a physical device, say a baseball bat, model gun etc… Six attached to a body created a simple but effective motion tracking system.

A developer panel was formed from; Alchemy VR, Rewind, Halo and the Mill – discussing use cases after a few years of practice and experience. Key questions were what did not work that included mainly thinking that you are creating a standard “framed” 2D/3D movie and old ideas fade to start and focus on specific POV object have to be rewritten. You can move but in sharper steps, and can cut when in context, although should contact some fixed reference point, and can experiment with exploration but may need to guide users for example turning down certain lights to allow the user to change their POV to the required place.

• Cool examples to search and download include the Everest exploration viewer – this takes the traditional vertigo effect seen in caves within immersive environments to a new cool location being half way up everest.
• ‘Home’ is the spacewalk that has parts of this that break good immersive experience (loosing a frame of reference) but that is what ‘…being in space is like’.
• For 360 video the one of the Great Barrier Reef is now a classic for its beauty and simplicity.
• HTC claimed over 1000 available just from last year and this will expand.

So can you do visualisation within this medium – will consider in detail later – but as an experience I would say there is scope to add this scale effect and immersion to help understand data. Couple of quick examples I would include;

• Understanding astronomy as is fully 3D and huge (see below from Manchester)
• Seeing the ATLAS detector and large underground building type structure showed its scale and shape

MT