Xylophone playing

in simulation & reality

Vision

it will be musical

https://youtu.be/aIvbzWrpqU4

This project focuses on making Roboy playing the xylophone in simulation and reality.

Our Goal

this semester we want…

move roboy’s arm & flick his hand in simulation & reality
let Roboy play the xylophone in simulation
teach Roboy to play the xylophone in reality
a hit detection in simulation & reality

Abstract

how we did it

Connections

  • Extended DartTracker

  • Tracks 3D model using Depth image

  • Requires initial defined object position

Simulation

  • Get Published TFs from Keys

  • Calculate inverse kinematics

  • Move to and try to hit a key

  • Wait for hit detection

Hit Detection in Simulation

  • Key joints experience acceleration when hit

  • Acceleration greater than a threshold → key has been hit

Results

roboy can now…

https://youtu.be/2PdDGQir7UM
Move his arm & flick his hand

Move his arm & flick his hand

in simulation & reality

Play the xylophone in simulation

Play the xylophone in simulation

The hit detection works in simulation

Play the xylophone in reality

Play the xylophone in reality

 

 

The Team

get to know the xylophone playing team

Team members SS2018

Simon Trendel (Team Lead)
Heather Tooill (Agile Coach)
Joana Pecholt
Niclas Wesemann
Muhammad Ansab Shohab
Stefan Walke

Links

codes, documentations & presentations

Where to go next

short-term & long-term

  • Fully automated state machine
  • Detect and Track objects
  • Hit detection in Real-life
  • Using both Hands
  • Human-like movements