Tuning Ultegra Di2

[Jeffr]

ok so, I got my new scope. I believe it is indeed differential phase shift keying at 1mhz. There seem to only be two phases that are not particularly cleanly modulated.

I was able to observe the phase transformations from multiple shifts and see different patterns depending on the shift that were identical when the same shift was repeated. Some time after the shift command is executed a follow-up position broadcast occurs multiple times. After properly setting a trigger I can capture any of these.

I think a high-pass filter and a pll would allow us to decode it. After that it'd be easier to reverse engineer the protocol once we're just dealing with binary.

[mgiaro]

hi Jeff, as usual a great job. Some times ago you told me arduino was too much power demanding for this application. Do yuo think the best way with ultegra is to make a new set of commands and brain? So that commands are simple buttons as in the dura ace?
Thanx so much anyway
Marcone

[Oilman]

Every low power micro controller will do. Atmega's can be bought in low power versions, so you could use arduino. But for example the TI MSP430 micro controllers are an obvious choice when it comes to a low power consumption. Shimano uses MSP430 uC in DA DI2.

[cwolves]

Newb EE question here since I'm a software engineer, but can anyone give me a direction on how to encode/decode a PSK signal? In terms of getting binary data out of the signal. I have a capture from a scope of the full communication that fires from a shift and can decode it by hand (would take a while), but I've no clue how to start decoding it electronically. I've a decent amount of experience with other electronics, but never messed with modulation over lines like this.

[mgiaro]

Hi electro-bikers...
no good news about hacking ultegra commands?

[Dr_Mutley]

keen to hear if there is a work around yet to eliminate the battery holder in tube mounted batteries?

[Deluz]

I am new to this thread and interested for several reasons:

I currently have Ultegra Di2 and would like to put the battery in the seatpost or elsewhere inside the frame or possibly under the saddle.

I hate the ergonomics of the Shimano Ultegra levers and would much prefer to use Campy levers with DIY buttons.

I could just take the guts out of the shifters and battery holder and be done with it but if I screw up I would be out of a lot of money and I would rather just sell them once I am done.
I am an EE with over 30 years of experience designing electronics including battery charging circuits.
I would be doing this as a fun project to accomplish my goals.
If I could end up with something marketable that would nice but I fear Shimano has patents on this and would crush anybody who tried to sell anything compatible with their Canbus products.
A few weeks ago I tried connected an oscilloscope to my Di2 harness. It was very difficult to hold the probe on the connector. I did see the DC power voltage but no modulated signal when operating the shifters for the rear.
I assume this was because I had the rear derailleur cable unplugged. Anyway I need to get a Di2 junction box and cable so I can hook the scope easily without disconnecting anything.
From what I have read so far the Canbus data is modulated on top of the DC power and might be phase shift encoded. I am not sure why they would use phase shift other than to make to it harder to reverse engineer.
I am also wondering if they encrypted the data which would not surprise me. This could make it really difficult to impossible.
I would think Shimano thought this through and would not make it easy because they would have a lot to loose if third parties started selling batteries and shifters.
Would like to hear any thoughts on this.

[Jeffr]

It did not look encrypted. It also didn't look like canbus. I just took a cable and cut it in the middle so I could hook the probes in-line wherever I wanted.

I believe what you will need to do is make a high-pass filter/amplifier to extract and boost the incoming signal. Then you will need a reasonably high quality oscillator and something like a PLL to extract a binary signal. The signal comes quite rapidly so you will also want some kind of register to feed it in parallel to a microcontroller to keep the clock rate down.

On the output side you will need a big enough cap to hold the juice to raise the line level for the signal as well as the reverse of the above. A register for the micro to write to, the pll and other output drivers.

On top of this you will need some collision detection logic. If you are going to intercept and modify signals as I wanted to you'd need all of this 2x in order to modify the signals in-line.

That's where I stopped. It is feasible but the number of parts involved is too great for me to handle solder reasonably.

[Deluz]

It did not look encrypted. It also didn't look like canbus. I just took a cable and cut it in the middle so I could hook the probes in-line wherever I wanted.

I believe what you will need to do is make a high-pass filter/amplifier to extract and boost the incoming signal. Then you will need a reasonably high quality oscillator and something like a PLL to extract a binary signal. The signal comes quite rapidly so you will also want some kind of register to feed it in parallel to a microcontroller to keep the clock rate down.

On the output side you will need a big enough cap to hold the juice to raise the line level for the signal as well as the reverse of the above. A register for the micro to write to, the pll and other output drivers.

On top of this you will need some collision detection logic. If you are going to intercept and modify signals as I wanted to you'd need all of this 2x in order to modify the signals in-line.

That's where I stopped. It is feasible but the number of parts involved is too great for me to handle solder reasonably.

Thanks for the info, it will give me a bit of a head start. Maybe it is or isn't Canbus but after decoding to binary data it will become more obvious.
Hopefully there is no encryption.
I just ordered a Di2 wire today and will plug into the spare port on the shifters to access the signal.

[Deluz]

Well I have looked at the signal with a scope and it is pretty much what you said.
The signal definitely looks like bpsk modulation.
I don't have any experience with this and have been doing some reading.
To demodulate the the signal it can be done in analog with phase lock loops, mixers, filters, etc or digitally with an A/D and FPGA or DSP processor.
I think I would prefer to take the digital route. In any case it won't be quick or easy.
I am a bit discouraged but not ready to give up yet.

[Jeffr]

The problem with a DSP or FPGA is of course power. That's why I mentioned registers on input/output to permit the micro to run at a low clock rate. Remember this system is always on. Always running. I suppose if it can wake up fast enough to still capture and demodulate the signal when it changes you could do something reasonable. I was doing that with interrupts on the avr.

[Deluz]

The problem with a DSP or FPGA is of course power. That's why I mentioned registers on input/output to permit the micro to run at a low clock rate. Remember this system is always on. Always running. I suppose if it can wake up fast enough to still capture and demodulate the signal when it changes you could do something reasonable. I was doing that with interrupts on the avr.

By registers I assume you mean something like a FIFO?
I guess that could work if there is enough idle time between packets to process the data at a lower speed.
The data rate is fairly high and there seems to a lot of data each time I press the shift button both during and after.
The carrier is 1 MHz and I think the data rate is 500 kbps

[Deluz]

I have made a bit of progress.
The DC level changes quite a bit from the battery voltage I assume from the motors in the derailleurs.
I made a simple high pass RC filter and it gets rid almost all of the DC and low frequency variations so I can just see the data.
Next step is put the waveform through a comparator to change it into square waves and then recover a clock.
Then I will be able to extract the the actual data.
I am wondering how sure anybody is that is actually CAN Bus protocol since Shimano's web site does not say it is.

[mgiaro]

Deluz you great!! And - by the way - new dura ace will be COMPATIBLE with the Ultegra...so your job will be even more important in the future.
I did several discover on old version, but...they are useless right now. The old dura ace di2 is going to desappear... ((

[Deluz]

Thanks for encouragement, it will help me to keep going.
There will be no quick solution, it will take some time.
I am hoping to hear from Oilman to see if I can get more info on the parts he has cracked open.