SkeltrackSkeleton

SkeltrackSkeleton — Object that tracks the joints in a human skeleton

Synopsis

struct              SkeltrackSkeleton;
struct              SkeltrackSkeletonClass;
SkeltrackSkeleton * skeltrack_skeleton_new              (void);
void                skeltrack_skeleton_track_joints     (SkeltrackSkeleton *self,
                                                         guint16 *buffer,
                                                         guint width,
                                                         guint height,
                                                         GCancellable *cancellable,
                                                         GAsyncReadyCallback callback,
                                                         gpointer user_data);
SkeltrackJointList  skeltrack_skeleton_track_joints_finish
                                                        (SkeltrackSkeleton *self,
                                                         GAsyncResult *result,
                                                         GError **error);
SkeltrackJointList  skeltrack_skeleton_track_joints_sync
                                                        (SkeltrackSkeleton *self,
                                                         guint16 *buffer,
                                                         guint width,
                                                         guint height,
                                                         GCancellable *cancellable,
                                                         GError **error);
void                skeltrack_skeleton_get_focus_point  (SkeltrackSkeleton *self,
                                                         gint *x,
                                                         gint *y,
                                                         gint *z);
void                skeltrack_skeleton_set_focus_point  (SkeltrackSkeleton *self,
                                                         gint x,
                                                         gint y,
                                                         gint z);

Object Hierarchy

  GObject
   +----SkeltrackSkeleton

Properties

  "dimension-reduction"      guint                 : Read / Write
  "enable-smoothing"         gboolean              : Read / Write
  "extrema-sphere-radius"    guint                 : Read / Write
  "graph-distance-threshold" guint                 : Read / Write
  "graph-minimum-number-nodes" guint                 : Read / Write
  "hands-minimum-distance"   guint                 : Read / Write
  "joints-persistency"       guint                 : Read / Write
  "shoulders-arc-length"     guint                 : Read / Write
  "shoulders-arc-start-point" guint                 : Read / Write
  "shoulders-circumference-radius" guint                 : Read / Write
  "shoulders-search-step"    gfloat                : Read / Write
  "smoothing-factor"         gfloat                : Read / Write
  "torso-minimum-number-nodes" gfloat                : Read / Write

Description

This object tries to detect joints of the human skeleton.

To track the joints, first create an instance of SkeltrackSkeleton using skeltrack_skeleton_new() and then set a buffer from where the joints will be retrieved using the asynchronous function skeltrack_skeleton_track_joints() and get the list of joints using skeltrack_skeleton_track_joints_finish().

A common use case is to use this library together with a Kinect device so an easy way to retrieve the needed buffer is to use the GFreenect library.

It currently tracks the joints identified by SkeltrackJointId .

Tracking the skeleton joints can be computational heavy so it is advised that the given buffer's dimension is reduced before setting it. To do it, simply choose the reduction factor and loop through the original buffer (using this factor as a step) and set the reduced buffer's values accordingly. The "dimension-reduction" property holds this reduction value and should be changed to the reduction factor used (alternatively you can retrieve its default value and use it in the reduction, if it fits your needs).

The skeleton tracking uses a few heuristics that proved to work well for tested cases but they can be tweaked by changing the following properties: "graph-distance-threshold" , "graph-minimum-number-nodes" , "hands-minimum-distance" , "shoulders-arc-start-point" , "shoulders-arc-length" , "shoulders-circumference-radius" , "shoulders-search-step" .

Details

struct SkeltrackSkeleton

struct SkeltrackSkeleton;


struct SkeltrackSkeletonClass

struct SkeltrackSkeletonClass {
  GObjectClass parent_class;
};

skeltrack_skeleton_new ()

SkeltrackSkeleton * skeltrack_skeleton_new              (void);

Constructs and returns a new SkeltrackSkeleton instance.

Returns :

The newly created SkeltrackSkeleton. [transfer full]

skeltrack_skeleton_track_joints ()

void                skeltrack_skeleton_track_joints     (SkeltrackSkeleton *self,
                                                         guint16 *buffer,
                                                         guint width,
                                                         guint height,
                                                         GCancellable *cancellable,
                                                         GAsyncReadyCallback callback,
                                                         gpointer user_data);

Tracks the skeleton's joints.

It uses the depth information contained in the given buffer and tries to track the skeleton joints. The buffer's depth values should be in mm. Use skeltrack_skeleton_track_joints_finish() to get a list of the joints found.

If this method is called while a previous attempt of tracking the joints is still running, a G_IO_ERROR_PENDING error occurs.

self :

The SkeltrackSkeleton

buffer :

The buffer containing the depth information, from which all the information will be retrieved.

width :

The width of the buffer

height :

The height of the buffer

cancellable :

A cancellable object, or NULL (currently unused). [allow-none]

callback :

The function to call when the it is finished tracking the joints. [scope async]

user_data :

An arbitrary user data to pass in callback, or NULL. [allow-none]

skeltrack_skeleton_track_joints_finish ()

SkeltrackJointList  skeltrack_skeleton_track_joints_finish
                                                        (SkeltrackSkeleton *self,
                                                         GAsyncResult *result,
                                                         GError **error);

Gets the list of joints that were retrieved by a skeltrack_skeleton_track_joints() operation.

Use skeltrack_joint_list_get_joint() with SkeltrackJointId to get the respective joints from the list. Joints that could not be found will appear as NULL in the list.

The list should be freed using skeltrack_joint_list_free().

self :

The SkeltrackSkeleton

result :

The GAsyncResult provided in the callback

error :

A pointer to a GError, or NULL. [allow-none]

Returns :

The SkeltrackJointList with the joints found. [transfer full]

skeltrack_skeleton_track_joints_sync ()

SkeltrackJointList  skeltrack_skeleton_track_joints_sync
                                                        (SkeltrackSkeleton *self,
                                                         guint16 *buffer,
                                                         guint width,
                                                         guint height,
                                                         GCancellable *cancellable,
                                                         GError **error);

Tracks the skeleton's joints synchronously.

Does the same as skeltrack_skeleton_track_joints() but synchronously and returns the list of joints found. Ideal for off-line skeleton tracking.

If this method is called while a previous attempt of asynchronously tracking the joints is still running, a G_IO_ERROR_PENDING error occurs.

The joints list should be freed using skeltrack_joint_list_free().

self :

The SkeltrackSkeleton

buffer :

The buffer containing the depth information, from which all the information will be retrieved.

width :

The width of the buffer

height :

The height of the buffer

cancellable :

A cancellable object, or NULL (currently unused). [allow-none]

error :

A pointer to a GError, or NULL. [allow-none]

Returns :

The SkeltrackJointList with the joints found. [transfer full]

skeltrack_skeleton_get_focus_point ()

void                skeltrack_skeleton_get_focus_point  (SkeltrackSkeleton *self,
                                                         gint *x,
                                                         gint *y,
                                                         gint *z);

Gets the focus point which is the origin from where the tracking will start. The coordinates will be in mm.

self :

The SkeltrackSkeleton

x :

The x coordinate of the focus point.

y :

The y coordinate of the focus point.

z :

The z coordinate of the focus point.

skeltrack_skeleton_set_focus_point ()

void                skeltrack_skeleton_set_focus_point  (SkeltrackSkeleton *self,
                                                         gint x,
                                                         gint y,
                                                         gint z);

Sets the focus point which is the origin from where the tracking will start. The coordinates should be in mm.

If this method is not called the default values are x = 0, y = 0, z = 1000, that is, in the center of the screen and at 1m from the camera.

self :

The SkeltrackSkeleton

x :

The x coordinate of the focus point.

y :

The y coordinate of the focus point.

z :

The z coordinate of the focus point.

Property Details

The "dimension-reduction" property

  "dimension-reduction"      guint                 : Read / Write

The value by which the dimension of the buffer was reduced (in case it was).

Allowed values: [1,1024]

Default value: 16


The "enable-smoothing" property

  "enable-smoothing"         gboolean              : Read / Write

Whether smoothing the joints should be applied or not.

Default value: TRUE


The "extrema-sphere-radius" property

  "extrema-sphere-radius"    guint                 : Read / Write

The radius of the sphere around the extremas (in mm).

Points inside this sphere are considered for calculating the average position of the extrema. If the value is 0, no averaging is done.

Allowed values: <= 65535

Default value: 300


The "graph-distance-threshold" property

  "graph-distance-threshold" guint                 : Read / Write

The value (in mm) for the distance threshold between each node and its neighbors. This means that a node in the graph will only be connected to another if they aren't farther apart then this value.

Allowed values: [1,65535]

Default value: 150


The "graph-minimum-number-nodes" property

  "graph-minimum-number-nodes" guint                 : Read / Write

The minimum number of nodes each of the graph's components should have (when it is not fully connected).

Allowed values: [1,65535]

Default value: 5


The "hands-minimum-distance" property

  "hands-minimum-distance"   guint                 : Read / Write

The minimum distance (in mm) that each hand should be from its respective shoulder.

Allowed values: >= 300

Default value: 550


The "joints-persistency" property

  "joints-persistency"       guint                 : Read / Write

The number of times that a joint can be null until its previous value is discarded. For example, if this property is 3, the last value for a joint will keep being used until the new value for this joint is null for 3 consecutive times.

Allowed values: <= 65535

Default value: 3


The "shoulders-arc-length" property

  "shoulders-arc-length"     guint                 : Read / Write

The length (in mm) of the arc where the shoulders will be searched. This length is used together with the SkeltrackSkeleton::shoulders-arc-start-point to determine the arc where the shoulders' points will be looked for.

Allowed values: [1,65535]

Default value: 250


The "shoulders-arc-start-point" property

  "shoulders-arc-start-point" guint                 : Read / Write

The starting point (in mm) of the arc (from the bottom of the shoulders' circumference) where the shoulders will be searched for. This point is used together with the SkeltrackSkeleton::shoulders-arc-length to determine the arc where the shoulders' points will be looked for.

Allowed values: [1,65535]

Default value: 100


The "shoulders-circumference-radius" property

  "shoulders-circumference-radius" guint                 : Read / Write

The radius of the circumference (in mm) from the head with which to look for the shoulders.

Allowed values: [1,65535]

Default value: 300


The "shoulders-search-step" property

  "shoulders-search-step"    gfloat                : Read / Write

The step considered for sampling the shoulders' circumference when searching for the shoulders.

Allowed values: [0.01,3.14159]

Default value: 0.01


The "smoothing-factor" property

  "smoothing-factor"         gfloat                : Read / Write

The factor by which the joints should be smoothed. This refers to Holt's Double Exponential Smoothing and determines how the current and previous data and trend will be used. A value closer to 0 will produce smoother results but increases latency.

Allowed values: [0,1]

Default value: 0.5


The "torso-minimum-number-nodes" property

  "torso-minimum-number-nodes" gfloat                : Read / Write

Minimum number of nodes for a component to be considered torso.

Allowed values: [0,65535]

Default value: 16