Top Diff List Farm Source Search Help RSS Login

HARK-KINECT Installation Instructions (as a USB recording device)

Softwares > HARK-KINECT > HARK-KINECT Installation Instructions (as a USB recording device)

HARK-KINECT Installation Instructions (as a USB recording device)


 Selection of Setup Environments


Nortifications for HARK-KINECT

  • Use USB2.0 ports rather than USB3.0 ports to connect a Kinect
  • Currently, only Kinect Xbox is supported (not Kinect Windows)
  • Installations for native ubuntu, namely CASE-I and CASE-II, are recommended.
  • Ubuntu 12.04 on VMware is not stable for USB device mount.

Following environments are supported by HARK-KINECT.

  • (CASE-I) Native 32bit Ubuntu 10.04 LTS or later
  • (CASE-II) Native 64bit Ubuntu 10.04 LTS or later
  • (CASE-III) VMware virtual machine {Host OS: Windows 7 (both 32bit and 64bit), Guest OS: Ubuntu 10.04 LTS or later (both 32bit and 64bit)}
  • (CASE-IV) VMware virtual machine {Host OS: Windows OS except Windows 7 (both 32bit and 64bit), Guest OS: Ubuntu 10.04 LTS or later (both 32bit and 64bit)}

Follow your appropriate section indicated by the items.
We do not support virtual machines in VMware Fusion working on Mac OSX.

 Installation for CASE-I


[Obsolete] Installation from pre-compiled binaries



Install the latest p7zip-full before the hark-kinect installation

wget "http://http.us.debian.org/debian/pool/main/p/p7zip/p7zip-full_9.20.1~dfsg.1-4_i386.deb"
sudo dpkg -i p7zip-full_9.20.1~dfsg.1-4_i386.deb

For adding the HARK repository, see HARK Installation Instructions.

hark-kinect installation

sudo apt-get install hark-kinect

Trouble shooting for pre-compiled binaries


Check if the device is connected by the following command.

cat /proc/asound/cards

Then you will see something like

0 [AudioPCI       ]: ENS1371 - Ensoniq AudioPCI
                     Ensoniq AudioPCI ENS1371 at 0x2080, irq 16
1 [Audio          ]: USB-Audio - Kinect USB Audio
                     Microsoft Kinect USB Audio at usb-0000:02:03.0-1, high speed
...

If you cannot see Microsoft Kinect USB Audio in the list, please try:

  • Reboot the machine
  • Check whether the USB port is USB2.0. Use USB2.0 not USB3.0
  • Check if your device is "before" the firmware update in Feburuary 2012 (Kinect after the firmware update enables Near-mode, which does not work with hark-kinect.)
  • Reinstall hark-kinect by the following commands
sudo apt-get --purge remove hark-kinect
sudo apt-get install hark-kinect

Installation from source compilation (Optional)


  • Supported OS
    • 32bit Ubuntu 10.04 or later

Install the latest p7zip-full before the hark-kinect installation

wget "http://http.us.debian.org/debian/pool/main/p/p7zip/p7zip-full_9.20.1~dfsg.1-4_i386.deb"
sudo dpkg -i p7zip-full_9.20.1~dfsg.1-4_i386.deb

For adding the HARK repository, see HARK Installation Instructions.

Create and move to your preferred directory to download source codes.

mkdir ${HOME}/YOUR_HARK_DOWNLOAD_DIR
cd ${HOME}/YOUR_HARK_DOWNLOAD_DIR

Download the latest hark-kinect-*.*.*.tar.gz from here and unzip it.

tar zxvf hark-kinect-*.*.*.tar.gz

Compile and install the source codes.

cd hark-kinect-*.*.*
sudo make install
sudo ./kinect_fetch_fw /lib/firmware/kinect

The "*.*.*" describes the package version.

Finally, reboot your machine.

sudo shutdown -r now

 Installation for CASE-II


[Obsolete] Installation from pre-compiled binaries



Install the latest p7zip-full before the hark-kinect installation

wget "http://http.us.debian.org/debian/pool/main/p/p7zip/p7zip-full_9.20.1~dfsg.1-4_amd64.deb"
dpkg -i p7zip-full_9.20.1~dfsg.1-4_amd64.deb

For adding the HARK repository, see HARK Installation Instructions.

hark-kinect installation

sudo apt-get install hark-kinect

Trouble shooting for pre-compiled binaries


Check if the device is connected by the following command.

cat /proc/asound/cards

Then you will see something like

0 [AudioPCI       ]: ENS1371 - Ensoniq AudioPCI
                     Ensoniq AudioPCI ENS1371 at 0x2080, irq 16
1 [Audio          ]: USB-Audio - Kinect USB Audio
                     Microsoft Kinect USB Audio at usb-0000:02:03.0-1, high speed
...

If you cannot see Microsoft Kinect USB Audio in the list, please try:

  • Reboot the machine
  • Check whether the USB port is USB2.0. Use USB2.0 not USB3.0
  • Check if your device is "before" the firmware update in Feburuary 2012 (Kinect after the firmware update enables Near-mode, which does not work with hark-kinect.)
  • Reinstall hark-kinect by the following commands
sudo apt-get --purge remove hark-kinect
sudo apt-get install hark-kinect

Installation from source compilation (Optional)


  • Supported OS
    • 64bit Ubuntu 10.04 or later

Install the latest p7zip-full before the hark-kinect installation

wget "http://http.us.debian.org/debian/pool/main/p/p7zip/p7zip-full_9.20.1~dfsg.1-4_amd64.deb"
dpkg -i p7zip-full_9.20.1~dfsg.1-4_amd64.deb

For adding the HARK repository, see HARK Installation Instructions.

Create and move to your preferred directory to download source codes.

mkdir ${HOME}/YOUR_HARK_DOWNLOAD_DIR
cd ${HOME}/YOUR_HARK_DOWNLOAD_DIR

Download the latest hark-kinect-*.*.*.tar.gz from here and unzip it.

tar zxvf hark-kinect-*.*.*.tar.gz

Compile and install the source codes.

cd hark-kinect-*.*.*
sudo make install
sudo ./kinect_fetch_fw /lib/firmware/kinect

The "*.*.*" describes the package version.

Finally, reboot your machine.

sudo shutdown -r now

 Installation for CASE-III


Supported Environment


  • Host OS
    • 32bit/64bit Windows 7
  • Guest OS
    • 32bit/64bit Ubuntu 10.04 LTS or later
  • Virtualization software (Use one of the following virtualization software.)
    • VMware workstation 8.0 or later
    • VMware player 4.0 or later

Installation


For Windows 7 OS, Microsoft provides an official SDK in the following website.


Close your VMware player/workstation before the Kinect driver installation.
Install the KinectSDK on your Host OS.
You can install the driver when you plug in your Kinect.

After the driver installation, open the VMware player/workstation.
Power on your virtual machine which you want to work with Kinect.
After booting the virtual machine, you can see

  • Microsoft Kinect USB Audio

on the VMware screen (VM > Removable Devices). The following figure is an example with VMware workstation 8.0.


Click "Connect (Disconnect from Host)" of the Microsoft Kinect USB Audio.

Then you can see the check mark on the Microsoft Kinect USB Audio like the following figure.


Trouble Shooting


If you cannot see "Microsoft Kinect USB Audio" on the VMware screen, the driver may not be correctly installed.

The most common failure is that the VMware shows only the motor device of the Kinect like the following figure.


Or, the Kinect audio device is sometimes mounted as "Microsoft Xbox NUI Audio" like the following figure.


However, this doesn't work properly in the virtual machine.
Check if the device name is "Microsoft Kinect USB Audio".

 Installation for CASE-VI


Supported Environment


  • Host OS
    • 32bit/64bit Windows OS except Windows 7
  • Guest OS
    • 32bit/64bit Ubuntu 10.04 LTS or later
  • Virtualization software (Use one of the following virtualization software.)
    • VMware workstation 8.0 or later
    • VMware player 4.0 or later

Installation Preparation on Ubuntu


Before installing the Kinect driver on Windows, we need a bit preparation on Ubuntu.

Type the following commands.

  • 32bit
wget "http://http.us.debian.org/debian/pool/main/p/p7zip/p7zip-full_9.20.1~dfsg.1-4_i386.deb"
sudo dpkg -i p7zip-full_9.20.1~dfsg.1-4_i386.deb
wget http://download.microsoft.com/download/F/9/9/F99791F2-D5BE-478A-B77A-830AD14950C3/KinectSDK-v1.0-beta2-x86.msi
7z e -y -r KinectSDK-v1.0-beta2-x86.msi "UACFirmware.*"
  • 64bit
wget "http://http.us.debian.org/debian/pool/main/p/p7zip/p7zip-full_9.20.1~dfsg.1-4_amd64.deb"
sudo dpkg -i p7zip-full_9.20.1~dfsg.1-4_amd64.deb
wget http://download.microsoft.com/download/F/9/9/F99791F2-D5BE-478A-B77A-830AD14950C3/KinectSDK-v1.0-beta2-x86.msi
7z e -y -r KinectSDK-v1.0-beta2-x86.msi "UACFirmware.*"

Then you will see a file named "UACFirmware.*" in the current directory.

Download the following link and unzip it.
hark_win_kinect.tar.gz

Put your "UACFirmware.*" file in the unzipped directory.
Now you have the following three files in the unzipped directory (hark-win-kinect).

  • "UACFirmware.*"
  • gen.sh
  • hark_win_kinect.exe

run gen.sh

./gen.sh

Then you will see "hark_win_kinect.bat" in the current directory.

Copy the whole directroy (hark-win-kinect) to the Windows OS.

Installation on Windows OS


Check if the hark-win-kinect directory created in Ubuntu was correctly copied to the Windows machine.

For Windows OS other than Windows 7, Microsoft does not provide an official SDK.
Therefore, we utilize a different driver based on libfreenect from the following website.


Download the source code of libfreenect to your preferred directory in Windows OS.
We need only the driver files in "platform/windows/inf" directory.

Close your VMware player/workstation before the Kinect driver installation.

Connect your Kinect sensor to your PC.
Your machine tries to install the driver automatically but it fails.
Install the driver on Windows OS using the driver files in "platform/windows/inf" in the libfreenect.

After the driver installation, you will see the following 3 devices in the Device Manager (Human Interface Device) on Windows OS.

  • Xbox NUI Audio
  • Xbox NUI Camera
  • Xbox NUI Motor

Open the VMware player/workstation.
Power on your virtual machine which you want to work with Kinect.
After booting the virtual machine, you can see

  • Xbox NUI Audio

on the VMware screen (VM > Removable Devices). The following figure is an example with VMware workstation 8.0.


DO NOT connect to the Xbox NUI Audio.

Now, double-click "hark-win-kinect.bat" in the hark-win-kinect directory.
You will see something like

About to send: 09 20 02 06 01 00 00 00 60 00 00 00 00 00 00 00 15 00 00 00 00 00 00 00
Reading first reply: 01 00 01 00 00 00 00 00 D4 07 00 00 01 00 01 00 00 00 00 00 D4 07 00 00 01 00 01 00 00 00 00 00 D4 07 00 00 FD FF FF FF 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
Reading reply: 00 E0 6F 0A 01 00 00 00 00 00 00 00
About to send: 09 20 02 06 02 00 00 00 00 40 00 00 03 00 00 00 00 00 08 00 00 00 00 00
...
Reading reply: 00 E0 6F 0A 0B 00 00 00 00 00 00 00

After the execution, see the VMware (VM > Removable Devices) screen again.
You will see that the "Xbox NUI Audio" is now renamed to "Microsoft Kinect USB Audio" like the following figure.


Click "Connect (Disconnect from Host)" of the Microsoft Kinect USB Audio.

Then you can see the check mark on the Microsoft Kinect USB Audio like the following figure.


Trouble Shooting


If you cannot see "Microsoft Kinect USB Audio" on the VMware screen, the driver may not be correctly installed.

The most common failure is that the VMware shows only the motor device of the Kinect like the following figure.


Or, the Kinect audio device is sometimes mounted as "Microsoft Xbox NUI Audio" like the following figure.


However, this doesn't work properly in the virtual machine.
Check if the device name is "Microsoft Kinect USB Audio".

 Device setup on Ubuntu


After the step in CASE-I/CASE-II/CASE-III/CASE-VI, you can use the Kinect device as ALSA device on Ubuntu.
Check if the device is connected by the following command.

cat /proc/asound/cards

Then you will see something like

0 [AudioPCI       ]: ENS1371 - Ensoniq AudioPCI
                     Ensoniq AudioPCI ENS1371 at 0x2080, irq 16
1 [Audio          ]: USB-Audio - Kinect USB Audio
                     Microsoft Kinect USB Audio at usb-0000:02:03.0-1, high speed
...

If you can see Microsoft Kinect USB Audio in the list.
The Kinect device is correctly mounted.

Notice that the number on the very left hand side is the sound card index.
In this example, the Kinect audio device is mounted as No. 1.

 How to use the Kinect device in HARK


We suppose you've read the HARK document in HARK Documentation page, and you have an intermediate level skill of HARK network file construction.

To use the Kinect device in HARK, put AudioStreamFromMic module on the network and set its parameters as follows:
Name Type Value Explanation
LENGTH int 512 Number of samples in one frame
ADVANCE int 160 Shift length
CHANNEL_COUNT int 4 Number of microphone channels
SAMPLING_RATE int 16000 Sampling rate of the audio stream
DEVICE_TYPE string ALSA Category of the sensors
DEVICE string plughw:* Device name

The asterisk in the DEVICE parameter means the sound card index.

cat /proc/asound/cards

provides the sound card index on the left hand side.
If you see

0 [AudioPCI       ]: ENS1371 - Ensoniq AudioPCI
                     Ensoniq AudioPCI ENS1371 at 0x2080, irq 16
1 [Audio          ]: USB-Audio - Kinect USB Audio
                     Microsoft Kinect USB Audio at usb-0000:02:03.0-1, high speed
...

set DEVICE=plughw:1 for using the Kinect device.