Since the end of November, I’ve been working full time on a new Solidity compiler written in rust, called Solang. Back in March 2019, I decided to prototype an idea: write a new Solidity compiler from scratch using llvm and rust. The prototype worked fine and I managed to get it integrated with Hyperledger Burrow; I was working at Monax at that time.
The existing Solidity Smart Contracts compiler has a number of problems. It’s a huge inscrutable C++ code base, and the language has all sorts of weird limitations. By starting from scratch we create a compiler which does not have these limitations. By using using llvm, we can also generate more optimal code. Lastly we can target different blockchains, not just ethereum.
I applied for a grant from the Web3 Foundation, and it was accepted. The grant covers language parity with the Ethereum Foundation Solidity compiler. So, over the next year, I’ll be working on Solang full time.
kernel v5.3 will introduce a new BPF feature: loops. This makes it possible to decode IR based on raw IR. This means the keymap does not list a specific protocol or protocol decoder; for each key it simply list the pulses and spaces that make up that key.
[[protocols]] name = 'BLAUPUNKT' protocol = 'raw' [[protocols.raw]] keycode = 'KEY_VIDEO_PREV' raw = '+6875 -6850 +660' [[protocols.raw]] keycode = 'KEY_VIDEO_NEXT' raw = '+8050 -8050 +660' [[protocols.raw]] keycode = 'KEY_UP' raw = '+3850 -3850 +660' [[protocols.raw]] keycode = 'KEY_OK' raw = '+7500 -7400 +660' [[protocols.raw]] keycode = 'KEY_DOWN' raw = '+4450 -4400 +660' [[protocols.raw]] keycode = 'KEY_RIGHT' raw = '+5660 -5630 +660' [[protocols.raw]] keycode = 'KEY_LEFT' raw = '+6280 -6200 +660' [[protocols.raw]] keycode = 'KEY_VOLUMEUP' raw = '+2650 -2580 +660' [[protocols.raw]] keycode = 'KEY_VOLUMEDOWN' raw = '+3250 -3200 +660' [[protocols.raw]] keycode = 'KEY_MUTE' raw = '+5050 -5000 +660' [[protocols.raw]] keycode = 'KEY_DOT' raw = '+2050 -2000 +660'
So for each key, in the raw string the + denotes a pulse and the - a space. The + and - prefixes are actually optional, since the position in the string determines whether it is pulse or space. To load these keymaps, you’ll need a git ir-keytable and kernel v5.3 or later.
You have a remote for which the is no rc keymap, but you do have a lircd.conf file which works. That requires running the lirc daemon; nowadays you most likely don’t need to any more.
Automagic conversion from lircd.conf to toml keymap
I’ve written a python script which parses a lircd.conf file, and outputs a toml file. Now this script should cover most cases, but it won’t work in all cases. Get lircd2toml.py from the v4l-utils contrib. This requires python3 to work.
Make lircd2toml.py executable: chmod 755 lircd2toml.py
Run ./lircd2toml.py -o output.toml yourlircdfile.lircd.conf
Once you have the output toml file, it probably needs a little fixing up. First of all, lircd does not usually use linux keycodes, so those will need adjusting. Have a look at the linux input keycodes for the list keycodes and adjust the toml accordingly. Also make sure the name is set to a reasonable value. Once you have done that, test it using:
ir-keytable -c -w output.toml -t
Please do let me know how you get on and comment below. Alternatively, find us on the linux-media mailinglist or on irc #v4l on Freenode. If the keymap works fine please submit it to the mailinglist as a patch for v4l-utils.
What if lircd2toml.py doesn’t work
First of all, please ask. I’m eager to hear about people using this tool. Having said that, I should explain a little more about lircd.conf files work.
IR decoders in a rc-core world
In the toml file there is the “protocol” field, which can either match a kernel decoder or BPF decoder.
There are two types of IR decoders in the rc-core world. There are the hardcoded decoders in the kernel, which cover the most common protocols: rc-5, nec, rc-6, rc-mm, sony, sanyo, jvc, sharp, xmp. If your remote uses one of these protocols, you can use the method described in the previous post. However, you can try the method with lircd2toml.py too, it should work for most cases.
There are also protocol decoders written in BPF, which also have parameters. So if your remote has a regular protocol with non-standard signaling length, you can specify that in your keymap, much like the parameters in lircd.conf files.
The protocol in the lircd.conf is set in the “flags” field. lircd has a few:
space_enc: This is anything where bits are encoded using the distance between to pulses or spaces. Many of these are plain nec, but not all. Those that are not, can be decoded using the BPF pulse_distance decoder, or BPF pulse_length depending on whether the space length differs or the pulse length differs.
space_first: Not sure yet.
rc5 or shift_enc: This is just rc-5. Some versions use non-standard times, in which case you should use the BPF manchester decoder.
rc6: This is just rc-6.
rcmm: This is just rc-mm.
grundig: This is for some ancient grundig remotes. There is a BPF grundig decoder for them.
bo: This is a Bang & Olufsen IR protocol. There is no BPF decoder for this yet, patches welcome :)
goldstar: This for some ancient goldstar remotes. There is no BPF decoder for this yet, and supported has been removed from lirc too
serial: This is for IR receivers that decode IR and put it on a tty. Not supported in rc-core, as this should be done by a device driver.
xmp: This is just xmp.
raw_codes: This is just a plain dump of the pulse/spaces for each button. This is not supported in BPF yet (but so below).
The parameters for the BPF decoders can be seen in the source code, see the global ints declared at the beginning of the c files. Have a look at the output of ir-ctl -r and see if you can match things up. Also have a look a the lirc documentation for the lircd format.
How to deal with raw_codes
If you have a lircd.conf remote with a raw_codes protocol, then things are a bit harder to solve unfortunately. The format lists all the keys and the associated IR, by listing the raw IR. The first value is a pulse, then space, pulse, etc. Essentially the protocol is not reverse engineered so you are not much better off than without the lird.conf. This is a case where you should reverse engineer the protocol, like you would if you had no lircd.conf file at all.
It might be possible to support raw_codes in BPF, but loops are not allowed so this is not trivial. You would have to create the state machine of walking through the list, and unrolling the loop enough so that it can successfully decode the IR.
Note that some raw_codes are a well-known protocol but the author of the lircd.conf did not realise. For example, this hauppauge lircd.conf file uses raw_codes but it’s actually rc-5.
If you get nothing, then unfortunately you’ll going to have to dig a little deeper. First of all verify that IR actually being received. Run:
And press some buttons again. If you still get nothing, check your batteries and that you have a clear line of sight between your remote and the IR receiver. If you do get something, then most likely you are unlucky and the protocol your remote uses is not one of the standard protocols supported by the linux kernel; a protocol decoder written in BPF would be in order, but that is the subject for another post.
For now I’m assuming that ir-keytable -c -p all -t gave you something like rc-5 or nec and no custom BPF decoder is needed.
Create the kernel rc keymap for your remote
The keymaps in /lib/udev/rc_keymmaps/ are written in toml, but they are generated from the kernel sources and you should create your keymap there so that it can be submitted as a patch for everyone else to use.
git clone the linux kernel tree and go to drives/media/rc/keymaps/. Copy one of the exists files to a new file, e.g. rc-foo.c and edit the file. First of the protocol should be set to one you discovered in the first step (e.g. RC_PROTO_NEC). Now you need to set the individual keys.
Using ir-keytable -c -p all -t you can get the scancode; that needs to be mapped to a linux keycode. All the keycodes are listed in include/uapi/linux/input-event-codes.h in the kernel tree. Note that there are entries for mouse buttons too.
Ensure you have the copyright and license header set correctly; also check you have created a new RC_MAP_FOO entry in include/media/rc-map.h and that your file is included in drivers/media/rc/keymaps/Makefile. Commit your changes.
In principle it is ready now to be submitted, but it should be tested of course.
Create the toml keymap for your keymap
In your cloned linux repo, ensure you can build the kernel and run it without errors
Do make sync-with-kernel KERNEL_DIR=path/to/your/linux/repo in v4l-utils
Now you should have an additional toml file in utils/keytable/rc_keymaps/
Load the keymap with ir-keytable -c -w foo.toml and test
Share your keymap with everyone else
The keymap should be submitted to linux-media kernel list, following all rules for submitting patches for the linux kernel. The patch should be against the kernel tree, and the v4l-utils tree will get updated by us once it’s merged into the linux-media git tree.
It has been a little quiet on the rc-core front, with just cosmetic changes in kernel v4.19 and v4.20. However, we do have two new features in kernel v5.0.
Mouse movement decoding in IR BPF
Some remotes have some sort directional pad or joystick which can be used as a mouse. When decoding using the lirc daemon, these are called lircmd.
From kernel v5.0 onwards the BPF function bpf_rc_pointer_rel(x, y) exists to report mouse movement. We use this to decode the iMON RSC remote protocol. This means that the SoundGraph iMON Station is now fully supported.
Driver for XBox DVD Remote
This is a usb device for the original XBox, and does not have a regular usb connector. You’ll need an adapter cable to make this work on non-XBox hardware.
This decoder only decodes IR from the remote the device comes with. I used the opportunity to figure out the protocol and now we have a decoder for this written in BPF, should you not have the decoder hardware.
Kernel v4.18 introduces a new type of BPF program, called BPF_PROG_LIRC_MODE2. This type of program can decode raw IR and report decoded scancodes. This is to support the many lircd.conf rc keymaps which are currently not supported by rc-core.
Now that the kernel space work is complete, ir-keytable needs to be extended to support BPF type decoders (loading, querying and deattaching) and we need a ne set of IR decoders. I’m currently working to this.
The ultimate goal is to support all the keymaps for IR decoding and sending that lircd currently supports, without the need to run a daemon.
Faster IR decoding
The in-kernel IR decoders have always been a little sluggish, since the timeouts they use are far greater than what is actually needed. So, now using IR is much more responsive and keys will less “sticky”. I think this makes a huge difference.
There some other minor improvements, such a MCE Keyboard decoder improvements, and when a lirc device is registered, it reports if is has a transmitter and if it is a raw or scancode receiver (or no receiver at all).
The rc_core_debug module parameter for the rc-core modules is gone. Debug must be enabled via dynamic debug.
There are minor fixes to ir-spi transmit and meson-ir timeout handling.
There is a new driver for the iMON Station, which is an external usb device (unlike most of the other iMON gear). It is a raw IR device, so it does no decoding in hardware.
The second new iMON feature is a decoder for the iMON PAD remotes. These remotes have their own protocol, which is decoded by the iMON Inside, iMON VFD or iMON Knob. I spent a fair amount of time decoding it, and I think this is the first time someone has figured out how it works. ir-keytable has be updated for this feature, which is not done yet. In the mean time, you can use it so:
Any mceusb device should have a wideband receiver for learning mode, which was not supported up until now. Learning mode should give you a more accurate reading, and can also measure the carrier. The downside is that the wideband receiver only works for short distances, so you have to hold your remote as close as possible to the IR receiver.
Using ir-ctl -m -r you can enable learning mode and carrier reports. The wideband receiver will remain enabled until you execute ir-ctl -M.
So you have your z8f0811 device from the ‘90s and you’ve misplaced the source code; what do you do? There is a simple method and a more complicated method if Read Protect was enabled during flash programming, using a Raspberry Pi as a flash programmer.
Using Debug PIN
The z8f0811 has debug pin, which controls the OCD interface (which has the descriptive name “On-Chip Debugger”. One of the commands is Read Program Memory (0BH). Since this is a one-pin interface, I connected an arduino and wrote the following program.
The Read Protect (RP) Option Bit does not prevent Flash access when bypassing the Flash Controller as described in ZiLOG Application Note AN0017 entitled Third Party Flash Programming Support of the Z8 Encore!® MCU. User code cannot be read through the On-Chip Debugger when read protect is enabled. User code can only be read out when bypassing the Flash Controller. Work-Around: None
My arduino did not have enough gpio pins, and I suspected that it couldn’t generate a clock signal (PWM) either. So I used a Raspberry Pi for this.
According to the documentation, the lowest clock rate the z8f0811 supports is 32KHz. So on the Raspberry Pi I use pwm to generate that clock; then I connect that to Xin on the z8f0811, and also to another gpio pin on the Raspberry Pi so I can wait for the rising edge.
Most other pins need connecting: debug for setting flash bypass, reset for resetting the device after the pwm clock is set up, and all of port A, B, C pins (11 pins).
We’ll need to bit-bang various pins so even at 32KHz, we need to be very careful about being timely. The only way to do this in Linux is by writing a kernel driver, and holding a spinlock (to avoid interrupts).
This is built against kernel 4.13-rc4 (simply because I was using that kernel version for testing RC core changes on the Raspberry Pi, and I knew that that version worked). This is using Fedora 26 (armhf).
After booting the kernel, you can copy the flash memory using:
dd if=/dev/zdumper of=z8f0811.bin bs=8192 count=1
Presumably suspect the other z8f08xx and z8f04xx devices will work fine too; they need the same pins connected. I have not tested this.
I realise this device is a microcontroller and not a microprocessor, but I can’t help but dislike the instruction set. Why are there so many addressing modes for 4kb of ram/registers? The disassembly is painful to read and write, as the same register can be access in many different ways.
There used to be a driver for this device in drivers/staging/media/lirc/lirc_sasem.c, but that was removed in kernel v4.12. I’ve since found it on eBay, and here is the result of analysing it, and I’ve written a new driver.
In the centre you can see the LCD, which is attached via a header. To the right you can see the IR receiver module. The LCD can be detached using the four screws. The LCD part is a NEC µPD16314 VFD (without a backlight). There is also a Cyruss CY7C63743 USB microcontroller. This drives the IR decoding and the LCD via its GPIO ports.
The NEC µPD16314 is a hd44780 compatible, which is ubiquitous and well documented. There is a driver for it in mainline, which can control it via gpio ports. So I tried to attach this LCD device to a Raspberry Pi via its gpio ports. This is easy to do with female-female jumper cables. After some measuring and experimenting, I got it to work.
The pinout of the LCD header is:
VCC - 5v not 3.3v
Not Connected (the pin is removed)
R/W (not needed, connect to ground)
Now, after compiling a kernel for the raspberry pi with CONFIG_HD44780 enabled, and the following added to the device tree:
So that worked! Using the CHARLCD driver, you can set a cursor position, cursor blinking, and reprogram character generator data. The original lirc_sasem driver could not do any of that!
Now that I know the pins of the LCD header, I can try to figure out what the Sasem Remote Controller sends it, and how that relates to usb packets we can send to it from the driver. The CY7C63743 is a OTP (One Time Program) device. It’s a device which makes it very easy to create keyboards, input devices, anything which is a low bandwidth usb device. Unfortunately I could find no way of dumping its program memory.
So, instead I attached a Saleae Logic 16 logic analyser to the board to see what commands it sends. I quickly discovered that it’s actually using the 4-bit interface, even though all 8 bits are connected via the header. Also, when the device powers on, it sends a few write reg commands and “Welcome DIGN Home Theatre PC”. During this period it drops any commands it receives over usb. In order to decode the commands, I wrote a simple libsigrokdecode decoder called sasem.
Since the CY7C63743 has 16 gpio ports, I don’t know why 4-bit mode is used. 1 gpio port is need for IR receive, one for the power button, and presumably another for switching the motherboard on (in case power on was pressed on the remote). That still leaves 13 gpio ports, and the LCD needs just 10 (8 bits data, Enable and Register Select).
The LCD has two commands: write data and write register. If you want to reprogram the display characters, you first do write register to DDRAM (display data) to set the location, followed by the letters you want using write data. When you attempt this via usb, it sends an extra “Write register DDRAM” before each data byte, which increases every time (from 0x80 to 0x8f for the first line and then 0xc0 to 0xcf for the second line). This makes it impossible to write to character generator ram (for your own characters), since every write data it sends a superfluous set DDRAM write register. I have no idea why this was done!
In addition, if you want to write to a register, you write 0x07 followed by the register value to usb. This means it is impossible to write 0x07 to the display. However this is one of the custom characters, for which we can’t write the graphics ram anyway.
The other oddity is if you do a write register DDRAM (to the location), then that command is passed on the device. However, as soon as you write data to the device, it gets preceeded by an annoying extra “write reg DDRAM” with an different address! So, the usb microcontroller maintains its own display position address, which you can modify using 0x09 0xYX (Y=0x4 for top, 0x0 for bottom) and X can be 0x1 to 0x10 (not base 0).
As for the Infrared interface, using ir-ctl I tried sending every protocol, and it reports data for NEC, Sanyo and JVC protocols. For some reason the JVC and Sanyo protocols comes out as semi-garbage, so the new driver only supports NEC.
Armed with this knowledge, I’ve created a new driver drivers/media/rc/sasem_ir.c. The driver can do a lot more than the original driver (e.g. cursor, blink) but no character graphics. Alas.
The iMon SoundGraph has very similar hardware, and I’ll be looking at that soon. I only have one model, if anyone can help with getting access to some more, that would be great.
rc-core supports a fair amount of devices, but not all are supported well. If there is any IR hardware which does not work, please send it to me and I will fix linux support if at all possible. I can send it back once the work is complete, else I can add it my collection of IR hardware.
The iMon SoundGraph support is fine, but some particular devices are not well supported. I’m sure the early iMon Knob most likely does not work; and judging by various posts there are others which don’t work either. Having access to those devices would be very helpful.
The SIR driver was ported from lirc without having hardware to test it on. It would be great if it could be verified it actually worked. I think the early Thinkpad T60 (like the Core Solo models) had a SIR IR port.
Remotes (IR protocol support)
Any remote which can send (or receive!) rc-mm, xmp, grundig, bang & olufsen protocol would be very interesting. I could use that to write IR decoders and encoders.