Oct 04, 2016

Pepper animation workflow improvement

A Unity application for controlling robot motion

We had our first encounter with Pepper at Junction Asia in Tokyo. During the Junction Asia hackathon, we came up with some cool animation ideas for the robot. However, we realized that the default tools used to create animations for Pepper were a bit cumbersome, and there would not be enough time to come up with anything useful at the hackathon. After the hackathon we joined the Pepper developer program and since then we have learned a lot about Pepper and its tools. In this blog entry we talk about our solution to improve Pepper’s animation features.

How animations are created for Pepper

Choregraphe (The Pepper’s development environment) has a timeline-based interface with keyframes and curves, similar to common animation programs. However, setting the robot’s pose for each keyframe is a slow process. When using a virtual robot, only forward kinematics (FK) are available. Setting values for each of the robot’s joints with a mouse is not very practical. A physical robot allows the user to manipulate the robot’s joints directly by twisting its arms, for example, but coming up with rich and realistic animations with this method is still very time consuming.

A human actor to speed-up the process

Professional animation workflows include, for example, motion capturing and blending together pre-recorded animations. Since we had some experience (Juha especially) with Kinect, we had an idea: could we use Kinect to track a human actor and convert the poses for Pepper?

Soon we found out that a guy named Atsushi Sugiyama from Japan had already done an implementation for Kinect v2. As we had a couple of older Kinects (v1) lying around, we tried to use those first with Unity, which has assets for real-time motion capture using Kinect. Later on we upgraded to Kinect v2 as v1 has inferior skeletal tracking and noisier output when compared to v2.

The problem with Kinect is the inaccurate tracking of hands and the head. Especially with Pepper, it is very important to track those parts accurately. A motion capture software for Kinect (and for many other devices, too) called iPi Soft supports PSMove controllers to improve head and hand tracking. This is a very inexpensive solution and fits perfectly to this case. Here is a neat example (made by HipsterSloth) of using PS Move API in Unity. The PS Move API removes the need for a PS console.

Our solution

We use Kinect v2 to generate a digital skeleton of a human actor. The skeletal data is sent to Unity and used as the skeleton for a humanlike virtual character, roughly the size of Pepper.

We also have a separate virtual model of Pepper. It has all the joints that Pepper has (as they are with their limitations) but not a skeletal humanlike structure. The Pepper model tries to mimic the poses of the virtual character using Inverse Kinematics (IK). This model’s joint data is then sent to NaoQI (robot’s OS) using UDP.

Choregraphe has tools for recording the motion of the robot (a physical or a virtual one). The recording can then be used as a base for the animation. As can be seen in the video, the system is quite useful already. We use it whenever we need to create custom animations for the robot.

Aug 03, 2016
Pepper is the most advanced humanoid robot currently available

Need an app for Pepper robot?

Coder Coded becomes Finland's first Pepper Developer Partner

We are proud to announce that we have become the first Finnish company to join the SoftBank Robotics Pepper Partners Program. We are well underway in developing our first apps, and we have been getting acquainted with our development unit for some time now.

What is Pepper?

Pepper robot is a human-shaped robot developed by SoftBank Robotics Europe. Its main application areas are based on human-robot communications and interaction. Pepper is a sympatethic and easily approachable humanoid robot that is currently used in customer service and households, for example. Initially, the robots do not have any applications in them, which is why developer partners are needed to develop the actual apps. At the time of writing, there are three Peppers in Finland alongside Coder Coded’s development robot.

Our goals regarding Pepper

In June 2016, the Finnish government outlined the use and development of intelligent robotics in Finland as one of the central objectives of the government program. Coder Coded’s goal is to advance the engineering knowledge of consumer oriented robotics application development, and to develop applications that will promote the adoption of Pepper robots in Finnish markets. Naturally, we also aim for international customers through the Pepper Partners Program.

Who we are

Coder Coded Inc is a Finnish IT-startup, which employs three persons. We have previously done e.g. exhibition installations for the Finnish science centre Heureka, as well as large customer projects for Finnish startups. We experiment by mixing modern web-technologies with the latest technological innovations. Thus, Pepper fits tremendously well to our development stack.

Inquiries and further information

Please, contact:

Mr Pekka Toiminen
[email protected]

For media:
Pepper images
Coder Coded logos

Nov 09, 2015

Our winning hack - Fear of Flying

Junction 2015 hackathon winner of .AERO track

Junction was one of the Slush hacks, held on November 6.-8. 2015 at Kattilahalli, Suvilahti in Helsinki. This hackathon had nine tracks to choose from. When the tracks were announced, we thought we might have a good idea for .AERO track by Finnair, Reaktor and Finavia. After signin up we didn’t plan it any further, just kept in mind that it could be something associated with fear of flying. Then we refined our idea during the hackathon.

Why this topic?

30% of people are anxious about flying (at least in some point of their lives). About 6% do not fly at all. Fear of flying or aviophobia is a problem that is recognized, but currently only a handful of innovations exist to cope with it. Some airlines provide hands-on courses a couple of times in a year to cope with aviophobia. These courses cost money, but are highly popular. We realized that there is practically zero content available for aviophobic people during flights.

How we planned to implement it?

New airplanes have a web server and a wi-fi access point for passengers. We designed a set of apps that utilize an accelerometer attached to the web server and the in-flight portal that provides the landing page for passengers using the airplane wi-fi. We have produced 4 different apps to give you an idea. Check out the slideshow online.

Turbulence meter

Airplanes can take much more than what people experience during turbulence. This meter displays the “safe zone” for airplane and plots the currently experienced g-forces. Demo online here, emulate accelerometer from devtools, or view with mobile device.

Mood meter

Passengers can now communicate their level of anxiety to the cabin crew by using the passenger mood-meter. It is embarrassing to talk directly to flight attendants about your fears when other people are listening. The mood-meter is a very subtle way to inform cabin crew about the level of anxiety of the passenger, and also the reason - another passenger makes you feel uncomfortable, or you’re afraid of current flight conditions. Check out the cabin crew view or passenger app demos online.

Cabin crew view shows a “seatmap heatmap” of passenger anxiety levels, making it possible for cabin crew to react in situations during flight.

Airplane soundtrack

A lot of people think that the airplane makes sounds that are not normal. This app plays the passenger all the normal sounds that airplane makes during flights. This way they can reassure themselves that the sounds they’re hearing are perfectly normal

Fun in the turbulence

We think that turbulence should be anticipated by the passengers. It is our job to come up with ideas that get into life only during turbulence. There could be games or similar that provide rewards during it, or something like we envisioned with wobbly faces / things. Couple of demos: Kekkonen and an NSFW version.

What next?

The winning hacks have a chance to pitch the ideas at Slush, so we’ll see you there!