Augmented Virtual Reality: Combining Crowd Sensing and Social Data Mining

Overview

Augmented reality is well known for extending the real world by adding computer-generated perceptual information and overlaid sensory information. In contrast, simulation worlds are commonly closed and rely on artificial sensory information generated by the simulator program or using data collected off-line. In this work, a new simulation paradigm is introduced providing augmented virtuality by integrating crowd sensing and social data mining in simulation worlds by using mobile agents. The simulation world interacts with real world environments, humans, machines, and other virtual worlds in real-time. Mobile agents are closely related to bots that can interact with humans via chat blogs. Among the mining of physical sensors (temperature, motion, position, light, ..), mobile agents can perform Crowd Sensing by participating in question-answer dialogs via a chat blog provided by a WEB App that can be used by the masses. Additionally, mobile agents can act as virtual sensors (offering data exchanged with other agents). Virtual sensors are sensor aggregators performing sensor fusion in a spatially region.

Content

Introduction to Augmented Virtuality

Main topic of this talk is Fusion of Real and Virtual worlds creating Augmented Virtuality by using Mobile Agents!

Simulation of Socio-Technical Systems

• Socio-technical systems are characterized by interactions of:

  • Human-Human (initiated by a human)
  • Human-Machine (initiated by a human)
  • Machine-Human (initiated by a machine, e.g., a chat bot)
  • Machine-Machine (initiated by a machine)

• The simulation of social ensemble behaviour requires simplification of interactions and individual behaviour

• Commonly simulations are performed with less than 1000 entities (humans, machines, ..) in a sandbox world

• Agent-based Modelling (ABM) is a suitable behaviour model for simulation

Augmented Reality

• Augmented reality is well known for extending the real world by adding computer-generated perceptual information and overlaid sensory information

[1]

Field Studies

• Experimental field studies are commonly used in social science to test social models or to derive social models
• The ensemble size in field studies is often limited to less than 1000 individuals or entities

Data Mining and Machine Learning are important tools to derive meaningful information from experimental and aggregated data.

Taxonomy of Data Mining

[3]

Crowd Sensing

• Crowd data can be used in field studies to extend the information data base or replace classical (survey) field studies

• Mobile Crowd Sensing combines aggregation of user data and mobile computing, i.e., creating spatially annotated data traces

• Among data supplied by users explicitly, sensor data of mobile devices can be used, too. But: Weakly correlated data!