10. RTOS Devices (Partitioning)

10.1. Overview

By default, all devices are assigned for non Real-time guests. To assign a specific device to Real-time guests, the RTOSVisor has to be configured appropriately. For each device, a device assignment script has to be created which assigns such a device to RTOS guests in general. In addition, a device configuration file for RTOS guests must be created which assigns such a device to a specific RTOS guest.

Caution

The current script files only support device assignment to a single RTOS guest.

10.2. PCI/PCIe Devices

Hint

This section is valid for all kinds of PCI/PCIe devices. For assigning PCI/PCIe Ethernet devices you should follow the steps described in PCI/PCIe Ethernet Devices

10.2.1. Detect pci bus info

The lspci command will present a list of all available pci devices. The first number sequence (e.g. 00:00.0) is the main part of the desired pci info. The number sequence is the vendor:device.function information. An additonal -D parameter shows the domain number.

sudo lspci -k

Here an example output:

hvuser@HV-TP106:~$ sudo lspci -k
00:00.0 Host bridge: Intel Corporation Xeon E3-1200 Processor Family DRAM Controller (rev 09)
        DeviceName:  Realtek 8211
        Subsystem: Fujitsu Technology Solutions Xeon E3-1200 Processor Family DRAM Controller
        Kernel driver in use: ie31200_edac
        Kernel modules: ie31200_edac
00:19.0 Ethernet controller: Intel Corporation 82579LM Gigabit Network Connection (Lewisville) (rev 04)
        Subsystem: Fujitsu Technology Solutions 82579LM Gigabit Network Connection (Lewisville)
        Kernel driver in use: e1000e
        Kernel modules: e1000e
00:1a.0 USB controller: Intel Corporation 6 Series/C200 Series Chipset Family USB Enhanced Host Controller #2 (rev 04)
        Subsystem: Fujitsu Technology Solutions 6 Series/C200 Series Chipset Family USB Enhanced Host Controller
        Kernel driver in use: ehci-pci
00:1c.0 PCI bridge: Intel Corporation 6 Series/C200 Series Chipset Family PCI Express Root Port 1 (rev b4)
        Kernel driver in use: pcieport
00:1c.6 PCI bridge: Intel Corporation 6 Series/C200 Series Chipset Family PCI Express Root Port 7 (rev b4)
        Kernel driver in use: pcieport
00:1c.7 PCI bridge: Intel Corporation 6 Series/C200 Series Chipset Family PCI Express Root Port 8 (rev b4)
        Kernel driver in use: pcieport
00:1d.0 USB controller: Intel Corporation 6 Series/C200 Series Chipset Family USB Enhanced Host Controller #1 (rev 04)
        Subsystem: Fujitsu Technology Solutions 6 Series/C200 Series Chipset Family USB Enhanced Host Controller
        Kernel driver in use: ehci-pci
00:1e.0 PCI bridge: Intel Corporation 82801 PCI Bridge (rev a4)
00:1f.0 ISA bridge: Intel Corporation C202 Chipset LPC Controller (rev 04)
        Subsystem: Fujitsu Technology Solutions C202 Chipset LPC Controller
        Kernel driver in use: lpc_ich
        Kernel modules: lpc_ich
00:1f.2 SATA controller: Intel Corporation 6 Series/C200 Series Chipset Family 6 port Desktop SATA AHCI Controller (rev 04)
        Subsystem: Fujitsu Technology Solutions 6 Series/C200 Series Chipset Family 6 port Desktop SATA AHCI Controller
        Kernel driver in use: ahci
        Kernel modules: ahci
00:1f.3 SMBus: Intel Corporation 6 Series/C200 Series Chipset Family SMBus Controller (rev 04)
        Subsystem: Fujitsu Technology Solutions 6 Series/C200 Series Chipset Family SMBus Controller
        Kernel driver in use: i801_smbus
        Kernel modules: i2c_i801
02:00.0 Ethernet controller: Intel Corporation 82574L Gigabit Network Connection
        Subsystem: Fujitsu Technology Solutions 82574L Gigabit Network Connection
        Kernel driver in use: e1000e
        Kernel modules: e1000e
03:00.0 VGA compatible controller: Matrox Electronics Systems Ltd. MGA G200e [Pilot] ServerEngines (SEP1) (rev 05)
        Subsystem: Fujitsu Technology Solutions MGA G200e [Pilot] ServerEngines (SEP1)
        Kernel driver in use: mgag200
        Kernel modules: mgag200

Example output with additional -D parameter:

hvuser@HV-TP106:~$ sudo lspci -kD
0000:00:00.0 Host bridge: Intel Corporation Xeon E3-1200 Processor Family DRAM Controller (rev 09)
        DeviceName:  Realtek 8211
        Subsystem: Fujitsu Technology Solutions Xeon E3-1200 Processor Family DRAM Controller
        Kernel driver in use: ie31200_edac
        Kernel modules: ie31200_edac
0000:00:19.0 Ethernet controller: Intel Corporation 82579LM Gigabit Network Connection (Lewisville) (rev 04)
        Subsystem: Fujitsu Technology Solutions 82579LM Gigabit Network Connection (Lewisville)
        Kernel driver in use: e1000e
        Kernel modules: e1000e
...

10.2.2. Case 1: PCIe Devices

In case a non-ethernet PCI Express (PCIe) device should be assigned to the real-time part, the supplied hv_addpcidev command should be used. The device <bus info> identified must be used as the first parameter of the hv_addpcidev command. The second (mandatory) parameter is a unique device name used on the Real-time side. The third (mandatory) parameter is a unique number (to be increased sequentially) for each assigned device, starting with 1.

hv_addpcidev 0000:02:00.0 rtos_pcidev1 1

This call creates two files: /hv/config/rtos_pcidev1.sh and /hv/config/rtos_pcidev1.config.

In case additional devices shall be assigned:

hv_addpcidev [DETECTED_BUS_INFO] rtos_pcidev2 2

Hint

The RTOS device name is mandatory. This parameter must be a unique name that is used to identify the device. This name will also be used in filenames that are created by the hv_addpcidev command. In this tutorial the default name used is rtos_pcidev1.

Caution

If devices with same name are assigned to the same Rtos the names of the keys in rtos_pcidev1.sh, rtos_pcidev1.config, etc. must be altered!

10.2.3. Case 2: Legacy PCI Devices

In case a non-ethernet Legacy PCI device should be assigned to the real-time part, the method described in this section has to be applied (all other information described for non-ethernet PCIe devices are valid, though). Compared with non-ethernet PCIe devices, a fourth parameter of the hv_addpcidev command is required.

sudo hv_addpcidev [DETECTED_BUS_INFO] rtos_pcidev1 1 <interrupt type>

Possible values of <interrupt type> are legacy or no_interrupt. If this parameter is not provided, the default MSI interrupt is used which is not feasible for Legacy PCI devices.

10.3. PCI/PCIe Ethernet Devices

Hint

All commands to be executed in the following guide have to be input via the shell. To open the shell right click on the desktop and select ‘Open Terminal here’ or press CRTL + ALT + T.

10.3.1. Device Identification

In a first step, it is required to determine the Ethernet device that shall be used by the Real-time guest. There are several ways how to detect the desired adapter.

10.3.1.1. Identify by hardware information

An easy way to identify an adapter is its hardware information:

lshw -class network

returns

*-network:1
    description: Ethernet interface
    product: 82545EM Gigabit Ethernet Controller (Copper)
    vendor: Intel Corporation
    physical id: 6
    bus info: pci@0000:02:06.0
    logical name: enp2s0
    version: 01
    serial: 00:0c:29:94:bb:c3
    size: 1Gbit/s
    capacity: 1Gbit/s
    width: 64 bits
    clock: 66MHz
    capabilities: bus_master cap_list rom ethernet physical logical tp 10bt 10bt-fd 100bt 100bt-fd 1000bt-fd autonegotiation
    configuration: autonegotiation=on broadcast=yes driver=e1000 driverversion=5.15.0-88-acontis duplex=full ip=172.17.10.26 latency=0 link=yes mingnt=255 multicast=yes port=twisted pair speed=1Gbit/s
    resources: irq:16 memory:fd580000-fd59ffff memory:fdfe0000-fdfeffff ioport:2080(size=64) memory:fd520000-fd52ffff

We can see much information helping on identification: The network adapter enp2s0 is an 'Intel' type '82545EM' with MAC-ID '00:0c:29:94:bb:c3' and current link state 'link=yes'.

Hint

If RT-Linux will be used as RTOS, then remember the currently used driver (in this example: driver=e1000), as this driver can also be used in RT-Linux with the attached device. This could be achieved by a modprobe [driver_name] call in the rtos console window.

10.3.2. Case 1: PCIe Ethernet Devices

In case a PCI Express (PCIe) device is assigned, the method described in this section has to be applied. The device name identified above must be used as the first parameter of the hv_addeth command. The second (mandatory) parameter is a unique device name used on the Real-time side. The third (mandatory) parameter is a unique number (to be increased sequentially) for each assigned device, starting with 1.

hv_addeth enp2s0 rtos_eth1 1

This call creates two files: /hv/config/rtos_eth1.sh and /hv/config/rtos_eth1.config.

In case additional network devices shall be assigned:

hv_addeth [DETECTED_NAME] rtos_eth2 2

Hint

The RTOS device name is mandatory. This parameter must be a unique name that is used to identify the device. This name will also be used in filenames that are created by the hv_addeth command. In this tutorial the default name used is rtos_eth1.

Caution

If devices with same name are assigned to the same Rtos the names of the keys in rtos_eth1.sh, rtos_eth1.config, etc. must be altered!

10.3.3. Case 2: Legacy PCI Ethernet Devices

In case a Legacy PCI device is assigned, the method described in this section has to be applied (all other information described for PCIe devices are valid, though). Compared with PCIe devices, a fourth parameter of the hv_addeth command is required.

sudo hv_addeth [DETECTED_NAME] rtos_eth 1 <interrupt type>

Possible values of <interrupt type> are legacy or no_interrupt. If this parameter is not provided, the default MSI interrupt is used which is not feasible for Legacy PCI devices.

10.4. Legacy Serial (COM) Devices (non PCI)

The first step is to get information about COM ports.

$ sudo dmesg | grep tty
ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A
Write it down somewhere: IRQ 4, IO PORT 0x3F8

The next step is to deactivate the COM port on the host side.

Adjust the files /etc/grub.d/40_custom and /etc/grub.d/41_custom.
You need to edit the linux kernel boot line which includes the text “linux /boot/vmlinuz- ” and add the following parameters to the end:
8250.nr_uarts=0
Save the file(s), update the grub menu and reboot.
sudo update-grub
sudo reboot
Now add the hardware information about the serial port to RTOSVisor. Create a new file with the name like /hv/config/rtos_com1.config and add the following lines:
RtosConfig
;-----------------------------------------------------------------------
; COM1 port configuration
;-----------------------------------------------------------------------
[Devices\Rtos\RTOS Serial port (COM1)]
"Type"=dword:0
"PciBus"=dword:0

[Devices\Rtos\RTOS Serial port (COM1)\Interrupt\1]
"Type"=dword:1
"Flags"=dword:3
"Id"=dword:4

[Devices\Rtos\RTOS Serial port (COM1)\IoPort\1]
"Address"=hex:F8,03,00,00,00,00,00,00
"Length"=dword:8
"Flags"=dword:11

;-----------------------------------------------------------------------
; End of file
;-----------------------------------------------------------------------
This configuration is valid for one Real-Time OS in your RTOSVisor. Please change the Devices\RtosX name if needed.
Please set Id to the interrupt number and Address is the IO PORT address you previously wrote down.

Edit the file usr.config and add a reference to the newly generated file at the end:
include "/hv/config/rtos_com1.config"
Now start the RT-Linux guest and take a look at the RT-Linux log. Find a line looking like
serial8250: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A
If you see this line, it means RT-Linux has recognized your COM port and uses it. The device name for this case is ttyS0.

You can test COM port with the command:
echo "Hello, other side!" > /dev/ttyS0

Hint

If your host computer has several serial ports, but you need to assign only one to RT-Linux, please use the last port.
For example, if your computer has port COM1 and COM2 and you want to use only one port in RT-Linux, change the grub kernel line accordingly: 8250.nr_uarts=1 and use COM2 for RT-Linux.

10.5. RTOS assignment (general)

To assign a specific device to Real-time guests in general, the partitioning script /hv/config/usr_hvpart.sh must be adjusted.

The device assignment scripts <RTOS device name>.sh usually shall be executed automatically on system startup. To accomplish this, you need to add the respective <RTOS device name>.sh calls into the file /hv/config/usr_hvpart.sh. In our tutorial we use rtos_eth1 as the unique <RTOS device name>, as mentioned earlier.

mousepad /hv/config/usr_hvpart.sh

The usr_hvpart.sh file should contain at least the following string after editing: source $HV_CONFIG/rtos_eth1.sh $cmd

The example below shows how the device with the unique name rtos_eth1 is assigned.

#!/bin/bash

cmd="add"
[ $1 == "delete" ] && cmd="delete"

# unbind devices (assign to RTOS)
source $HV_CONFIG/rtos_eth1.sh $cmd

Please run the hv_hvpart command with the parameter add or reboot the system to make the change effective.

hv_hvpart add

You may use the delete parameter to assign all RTOS devices back to the Hypervisor Host.

hv_hvpart delete

Hint

The $HV_BIN/hvpart.sh script will be automatically started via the systemd service controlled via /etc/systemd/system/hv_part.service. This script will call the usr_hvpart.sh script which includes user specific partitioning commands. This service can be enabled or disabled as shown below (by default, it is enabled)

sudo systemctl enable /hv/services/hv_part.service
sudo systemctl disable hv_part

Caution

The System Manager configuration tool will write the device assignment into /hv/bin/hvpart.sh, you must not change this file. Effectively, all devices listed in /hv/bin/hvpart.sh and /hv/config/usr_hvpart.sh will be assigned to the RTOS guests.

10.5.1. Verification

10.5.1.1. de-assignment check using Hypervisor Host

In a first step, check if the Hypervisor Host’s original driver is not used in conjunction with the devices assigned to the Real-time guest.

lspci -k

The output will look similar like the following excerpt:

    :         :        :         :        :         :        :         :        :         :
    :         :        :         :        :         :        :         :        :         :
    :         :        :         :        :         :        :         :        :         :
01:00.0 Ethernet controller: Intel Corporation I210 Gigabit Network Connection (rev 03)
        Subsystem: Intel Corporation I210 Gigabit Network Connection
        Kernel driver in use: igb
        Kernel modules: igb
02:00.0 Ethernet controller: Intel Corporation I210 Gigabit Backplane Connection (rev 03)
        Subsystem: Intel Corporation I210 Gigabit Backplane Connection
        Kernel driver in use: pci-stub
        Kernel modules: igb
03:00.0 Ethernet controller: Intel Corporation I210 Gigabit Backplane Connection (rev 03)
        Subsystem: Intel Corporation I210 Gigabit Backplane Connection
        Kernel driver in use: pci-stub
        Kernel modules: igb
    :         :        :         :        :         :        :         :        :         :
    :         :        :         :        :         :        :         :        :         :
    :         :        :         :        :         :        :         :        :         :

In the above example, the instance 01:00.0 is used by Ubuntu (driver: igb, e1000e etc.) and the instances 02:00.0 and 03:00.0 are assigned to a Real-time guest (driver: pci-stub).

10.6. Assign a device to a specific Real-time guest

After creating the device configuration file <RTOS device name>.config, it needs to be included into the guest configuration file to become effective for the respective guest. The next steps describe how to add this device to the example Real-time Linux guest, we use rtos_eth1 as the unique <RTOS device name>.

Hint

Initially, the Hypervisor Host does not provide any example guest folders. To switch to an example guest, you must execute the corresponding initialization. For instructions on how to initialize the examples, refer to Example guest folders.

hv_open_example rt-linux
hv_sync_example rt-linux
cd /hv/guests/guestrtlinux
mousepad ./usr.config

Add the corresponding rtos_eth1.config entry to the includes section of the configuration file.

The following example shows the ‘modified’ usr.config file:

RtosConfig
;-----------------------------------------------------------------------
; acontis technologies GmbH
;
; Guest user configuration
;-----------------------------------------------------------------------

#include "/hv/config/rtos_eth1.config"

;-----------------------------------------------------------------------
; End of file
;-----------------------------------------------------------------------

10.6.1. Real-time guest assignment check

10.6.1.1. Real-time Linux

After assigning a device to the RT-Linux a driver could be loaded for that device.

Run the rtos:

cd /hv/guests/guestrtlinux
hv_guest_start -view

Log in into Real-Time Linux and run modprobe [driver_name]:

vmf64 login: root
password:  root
modprobe e1000

Hint

In the default case the previously used Hypervisor Host driver can still be used in RT-Linux.

10.6.1.2. RTOS-32

We use the RTOS-32Demo application to quickly verify, if the assigned netwok adapter is visible to the rtos part. First, the rtos32 example must be initialized and the usr.config file needs to be adjusted.

hv_open_example rtos-32
hv_sync_example rtos-32
cd /hv/guests/guestrtos32
mousepad ./usr.config

Add the corresponding rtos_eth1.config entry to the includes section of the configuration file.

The following example shows the ‘modified’ usr.config file:

RtosConfig
;-----------------------------------------------------------------------
; acontis technologies GmbH
;
; Guest user configuration
;-----------------------------------------------------------------------

#include "/hv/config/rtos_eth1.config"

;-----------------------------------------------------------------------
; End of file
;-----------------------------------------------------------------------

Adjust the guest configuration setting to prepare starting the RTOS-32Demo:

cd /hv/guests/guestrtos32
mousepad usr_guest_config.sh

and add the following line:

export osImage=$HV_ROOT/guests/guestrtos32/Loader.bin

Adjust the link to the demo application:

cd /hv/guests/guestrtos32
rm rtos32app.dlm
ln -s /hv/guests/guestrtos32/files/RTOS-32Demo.dlm rtos32app.dlm

Run the demo:

cd /hv/guests/guestrtos32
hv_guest_start -view

The output should look like this for the used network adapter (8086:1533) in this tutorial:

PCI BIOS Information

Vendor Device Bus Dev Func Class  Int IRQ TLat MSI Type
-------------------------------------------------------
FFFE   FFFE     0   0   0  000000  -    0    0  -  Invalid class code
FFFE   FFFE     0   2   0  000000  -    0    0  -  Invalid class code
FFFE   FFFE     0  20   0  000000  -    0    0  -  Invalid class code
FFFE   FFFE     0  22   0  000000  -    0    0  -  Invalid class code
FFFE   FFFE     0  26   0  000000  -    0    0  -  Invalid class code
FFFE   FFFE     0  27   0  000000  -    0    0  -  Invalid class code
FFFE   FFFE     0  28   0  000000  -    0    0  -  Invalid class code
FFFE   FFFE     0  29   0  000000  -    0    0  -  Invalid class code
FFFE   FFFE     0  30   0  000000  -    0    0  -  Invalid class code
FFFE   FFFE     0  31   0  000000  -    0    0  -  Invalid class code
8086   1533     1   0   0  020000  A    5    0  X  Network controller
FFFE   FFFE     2   0   0  000000  -    0    0  -  Invalid class code
10EC   8169     3   5   0  020000  A    4   64  -  Network controller
Current Date/time: 01/01/2018 00:00:00
Current Date/time: 01/01/2018 00:00:01
Current Date/time: 01/01/2018 00:00:02
Current Date/time: 01/01/2018 00:00:03
Current Date/time: 01/01/2018 00:00:04

List of threads:
Name            Prio State      R.Del FStck  MStck Scheds  CTime  CT%
---------------------------------------------------------------------
Main Task          5 Current          63060  61276     11  -      -
Idle Task          0 Ready             2164   2164    526  -      -
Comm               5 Delaying       1 32832  31724    527  -      -
IPTASK             6 BlckdWait         4128   3940      2  -      -
IPTIMER            6 Delaying      11  4120   3940     44  -      -

Interrupts:
 IRQ    Calls  FreeStack  Doubles  Time
-------------------------------------------
   0      526        744        0  -

Network:
 Ping will respond at IP address 192.168.157.2

In the above example, one 8086:1533 device at BDF 1,0,0 is assigned to the Real-time guest.

Finally, terminate the console connection to the real-time guest and stop the Real-time guest OS:

CTRL + C

cd /hv/guests/guestrtos32
hv_guest_stop

10.7. RTOS de-assignment

To de-assign a specific device from Real-time guests, the partitioning script /hv/config/usr_hvpart.sh must be adjusted.

In a first step, please de-assign all devices from RTOS guests using the delete parameter of the hv_hvpart command (assign all RTOS devices back to the Hypervisor Host):

hv_hvpart delete

Then you need to remove or uncomment the respective <RTOS device name>.sh calls from the file /hv/config/usr_hvpart.sh. Here we use rtos_eth1 as the unique <RTOS device name>, as mentioned earlier.

mousepad /hv/config/usr_hvpart.sh

The usr_hvpart.sh file should contain at least the following string after editing: source $HV_CONFIG/rtos_eth1.sh $cmd

The example below shows how the device with the unique name rtos_eth1 is assigned.

#!/bin/bash

cmd="add"
[ $1 == "delete" ] && cmd="delete"

# unbind devices (assign to RTOS)
# source $HV_CONFIG/rtos_eth1.sh $cmd       --> uncomment or remove this line

Finally, run the hv_hvpart command with the parameter add or reboot the system to assign the remaining devices to RTOS guests again.

hv_hvpart add