Wiki

This version (09 Nov 2020 11:01) was approved by mhennerich.The Previously approved version (09 Nov 2020 10:42) is available.Diff

ADRV9009-ZU11EG Multi-SOM Synchronization

….

Hardware setup

Required equipment

EVAL-HMC7044 modifications

There is no over-voltage or reverse polarity protection

Connect a 5V 1A power supply to TP17(GND) and TP14(VCC)

The following changes need to be made on the evaluation board:

  1. Populate J4 and J20 SMA
  2. Replace C28 and C59 with 0 ohm resistors
  3. Replace R159,160,276,180,181,365 with 50 ohm resistors

SPI connection to the carrier board

The following connections between connectors P25(carrier) and J1(clk) need to be done.

P25 J1 Function
2 12 GND
3 20 CS
4 16 MOSI
5 18 MISO
6 14 CLK

RefCLK and Sync connections to the carrier boards

  1. CLKOUT5_P and CLKOUT6_P connect to the SYNC SMAs on the carrier
  2. CLKOUT0 and CLKOUT2 connect to the REFCLK SMAs on the carrier

Software

The board which connects the external HMC7044 clockchip is refered as primary and requires the devicetree (system.dtb) from the primary folder in the archive. Likewise the board without the external clockchip connected is called secondary, and requires the devicetree from the secondary folder.

Theory of Operation

There are two domains of synchronization that are considered in this configuration, the ADRV9009 transceivers and the FPGAs. Synchronization for the transceivers is provided by the clocking tree of HMC7044s and the JESD protocol. In the diagram several HMC7044s are cascade from a parent, or what we call external HMC7044, who is responsible for general system reference (sysref) control. These reference signals feed the clock chips on the individual SOMs and FPGAs.

talise_clocking_tree

During multi-chip synchronization (MCS), which is a feature of the ADRV9009s, all baseband data from the converters is synchronized across transceiver chips. This requires specific sysrefs to be captured at each of the transceiver simultaneously. This will also create deterministic phase differences between transceivers, when RFPLL sync is enabled, as well. The individual API rights to the transceivers, clock chips, and their sequences are detailed in the rx method of the python class adrv9009_zu11eg_multi.py.

For more information on:

Please visit the JESD204 (FSM) Interface Linux Kernel Framework page

Running the software

examples/adrv9009_som_multi.py

michael@mhenneri-D06:~/devel/git/pyadi-iio$ python3 examples/adrv9009_som_multi.py 
--Connecting to devices


Iteration# 0
ip:10.44.3.53: DEVICE0: Is <Paused> in state <clk_sync_stage1> with status <0>
ip:10.44.3.39: DEVICE1: Is <Paused> in state <clk_sync_stage1> with status <0>
ip:10.44.3.53: DEVICE0: Is <Paused> in state <clk_sync_stage2> with status <0>
ip:10.44.3.39: DEVICE1: Is <Paused> in state <clk_sync_stage2> with status <0>
ip:10.44.3.53: DEVICE0: Is <Paused> in state <clk_sync_stage3> with status <0>
ip:10.44.3.39: DEVICE1: Is <Paused> in state <clk_sync_stage3> with status <0>
ip:10.44.3.53: DEVICE0: Is <Paused> in state <link_setup> with status <0>
ip:10.44.3.39: DEVICE1: Is <Paused> in state <link_setup> with status <0>
ip:10.44.3.53: DEVICE0: Is <Paused> in state <opt_setup_stage1> with status <0>
ip:10.44.3.39: DEVICE1: Is <Paused> in state <opt_setup_stage1> with status <0>
ip:10.44.3.53: DEVICE0: Is <Paused> in state <opt_setup_stage2> with status <0>
ip:10.44.3.39: DEVICE1: Is <Paused> in state <opt_setup_stage2> with status <0>
ip:10.44.3.53: DEVICE0: Is <Paused> in state <opt_setup_stage3> with status <0>
ip:10.44.3.39: DEVICE1: Is <Paused> in state <opt_setup_stage3> with status <0>
ip:10.44.3.53: DEVICE0: Is <Paused> in state <opt_setup_stage4> with status <0>
ip:10.44.3.39: DEVICE1: Is <Paused> in state <opt_setup_stage4> with status <0>
ip:10.44.3.53: DEVICE0: Is <Paused> in state <opt_setup_stage5> with status <0>
ip:10.44.3.39: DEVICE1: Is <Paused> in state <opt_setup_stage5> with status <0>
ip:10.44.3.53: DEVICE0: Is <Paused> in state <clocks_enable> with status <0>
ip:10.44.3.39: DEVICE1: Is <Paused> in state <clocks_enable> with status <0>
ip:10.44.3.53: DEVICE0: Is <Paused> in state <link_enable> with status <0>
ip:10.44.3.39: DEVICE1: Is <Paused> in state <link_enable> with status <0>
HMC7044s CAP bank select:  [14, 14, 14, 13, 13, 14, 13]
JESD Link status: DATA (84a50000.axi-jesd204-rx)
JESD Link status: DATA (84a30000.axi-jesd204-tx)
JESD Link status: DATA (84a70000.axi-jesd204-rx)
JESD Link status: DATA (84a50000.axi-jesd204-rx)
JESD Link status: DATA (84a30000.axi-jesd204-tx)
JESD Link status: DATA (84a70000.axi-jesd204-rx)
JESD SYSREF captured: Yes (84a50000.axi-jesd204-rx)
JESD SYSREF captured: Yes (84a30000.axi-jesd204-tx)
JESD SYSREF captured: Yes (84a70000.axi-jesd204-rx)
JESD SYSREF captured: Yes (84a50000.axi-jesd204-rx)
JESD SYSREF captured: Yes (84a30000.axi-jesd204-tx)
JESD SYSREF captured: Yes (84a70000.axi-jesd204-rx)
JESD SYSREF alignment error: No (84a50000.axi-jesd204-rx)
JESD SYSREF alignment error: No (84a30000.axi-jesd204-tx)
JESD SYSREF alignment error: No (84a70000.axi-jesd204-rx)
JESD SYSREF alignment error: No (84a50000.axi-jesd204-rx)
JESD SYSREF alignment error: No (84a30000.axi-jesd204-tx)
JESD SYSREF alignment error: No (84a70000.axi-jesd204-rx)
JESD 84a50000.axi-jesd204-rx:  0 0 0 0 0 0 0 0
JESD 84a70000.axi-jesd204-rx:  0 0 0 0 0 0 0 0
JESD 84a50000.axi-jesd204-rx:  0 0 0 0 0 0 0 0
JESD 84a70000.axi-jesd204-rx:  0 0 0 0 0 0 0 0
JESD 84a50000.axi-jesd204-rx:  Yes Yes Yes Yes Yes Yes Yes Yes
JESD 84a70000.axi-jesd204-rx:  Yes Yes Yes Yes Yes Yes Yes Yes
JESD 84a50000.axi-jesd204-rx:  Yes Yes Yes Yes Yes Yes Yes Yes
JESD 84a70000.axi-jesd204-rx:  Yes Yes Yes Yes Yes Yes Yes Yes
JESD 84a50000.axi-jesd204-rx:  Yes Yes Yes Yes Yes Yes Yes Yes
JESD 84a70000.axi-jesd204-rx:  Yes Yes Yes Yes Yes Yes Yes Yes
JESD 84a50000.axi-jesd204-rx:  Yes Yes Yes Yes Yes Yes Yes Yes
JESD 84a70000.axi-jesd204-rx:  Yes Yes Yes Yes Yes Yes Yes Yes
###########
Across Chip (A):	 16.733179
Across FMC8 (A):	 174.713495
Across Chip (B):	 8.009678
Across FMC8 (B):	 -162.783896
Across SoM (AB):	 40.307809
###########


Iteration# 1
ip:10.44.3.53: DEVICE0: Is <Paused> in state <clk_sync_stage1> with status <0>
ip:10.44.3.39: DEVICE1: Is <Paused> in state <clk_sync_stage1> with status <0>
ip:10.44.3.53: DEVICE0: Is <Paused> in state <clk_sync_stage2> with status <0>
ip:10.44.3.39: DEVICE1: Is <Paused> in state <clk_sync_stage2> with status <0>
ip:10.44.3.53: DEVICE0: Is <Paused> in state <clk_sync_stage3> with status <0>
ip:10.44.3.39: DEVICE1: Is <Paused> in state <clk_sync_stage3> with status <0>
ip:10.44.3.53: DEVICE0: Is <Paused> in state <link_setup> with status <0>
ip:10.44.3.39: DEVICE1: Is <Paused> in state <link_setup> with status <0>
ip:10.44.3.53: DEVICE0: Is <Paused> in state <opt_setup_stage1> with status <0>
ip:10.44.3.39: DEVICE1: Is <Paused> in state <opt_setup_stage1> with status <0>
ip:10.44.3.53: DEVICE0: Is <Paused> in state <opt_setup_stage2> with status <0>
ip:10.44.3.39: DEVICE1: Is <Paused> in state <opt_setup_stage2> with status <0>
ip:10.44.3.53: DEVICE0: Is <Paused> in state <opt_setup_stage3> with status <0>
ip:10.44.3.39: DEVICE1: Is <Paused> in state <opt_setup_stage3> with status <0>
ip:10.44.3.53: DEVICE0: Is <Paused> in state <opt_setup_stage4> with status <0>
ip:10.44.3.39: DEVICE1: Is <Paused> in state <opt_setup_stage4> with status <0>
ip:10.44.3.53: DEVICE0: Is <Paused> in state <opt_setup_stage5> with status <0>
ip:10.44.3.39: DEVICE1: Is <Paused> in state <opt_setup_stage5> with status <0>
ip:10.44.3.53: DEVICE0: Is <Paused> in state <clocks_enable> with status <0>
ip:10.44.3.39: DEVICE1: Is <Paused> in state <clocks_enable> with status <0>
ip:10.44.3.53: DEVICE0: Is <Paused> in state <link_enable> with status <0>
ip:10.44.3.39: DEVICE1: Is <Paused> in state <link_enable> with status <0>
HMC7044s CAP bank select:  [14, 14, 14, 13, 13, 14, 13]
JESD Link status: DATA (84a50000.axi-jesd204-rx)
JESD Link status: DATA (84a30000.axi-jesd204-tx)
JESD Link status: DATA (84a70000.axi-jesd204-rx)
JESD Link status: DATA (84a50000.axi-jesd204-rx)
JESD Link status: DATA (84a30000.axi-jesd204-tx)
JESD Link status: DATA (84a70000.axi-jesd204-rx)
JESD SYSREF captured: Yes (84a50000.axi-jesd204-rx)
JESD SYSREF captured: Yes (84a30000.axi-jesd204-tx)
JESD SYSREF captured: Yes (84a70000.axi-jesd204-rx)
JESD SYSREF captured: Yes (84a50000.axi-jesd204-rx)
JESD SYSREF captured: Yes (84a30000.axi-jesd204-tx)
JESD SYSREF captured: Yes (84a70000.axi-jesd204-rx)
JESD SYSREF alignment error: No (84a50000.axi-jesd204-rx)
JESD SYSREF alignment error: No (84a30000.axi-jesd204-tx)
JESD SYSREF alignment error: No (84a70000.axi-jesd204-rx)
JESD SYSREF alignment error: No (84a50000.axi-jesd204-rx)
JESD SYSREF alignment error: No (84a30000.axi-jesd204-tx)
JESD SYSREF alignment error: No (84a70000.axi-jesd204-rx)
JESD 84a50000.axi-jesd204-rx:  0 0 0 0 0 0 0 0
JESD 84a70000.axi-jesd204-rx:  0 0 0 0 0 0 0 0
JESD 84a50000.axi-jesd204-rx:  0 0 0 0 0 0 0 0
JESD 84a70000.axi-jesd204-rx:  0 0 0 0 0 0 0 0
JESD 84a50000.axi-jesd204-rx:  Yes Yes Yes Yes Yes Yes Yes Yes
JESD 84a70000.axi-jesd204-rx:  Yes Yes Yes Yes Yes Yes Yes Yes
JESD 84a50000.axi-jesd204-rx:  Yes Yes Yes Yes Yes Yes Yes Yes
JESD 84a70000.axi-jesd204-rx:  Yes Yes Yes Yes Yes Yes Yes Yes
JESD 84a50000.axi-jesd204-rx:  Yes Yes Yes Yes Yes Yes Yes Yes
JESD 84a70000.axi-jesd204-rx:  Yes Yes Yes Yes Yes Yes Yes Yes
JESD 84a50000.axi-jesd204-rx:  Yes Yes Yes Yes Yes Yes Yes Yes
JESD 84a70000.axi-jesd204-rx:  Yes Yes Yes Yes Yes Yes Yes Yes
###########
Across Chip (A):	 16.754098
Across FMC8 (A):	 174.305087
Across Chip (B):	 7.621835
Across FMC8 (B):	 -163.603793
Across SoM (AB):	 40.224082
###########


Iteration# 2
ip:10.44.3.53: DEVICE0: Is <Paused> in state <clk_sync_stage1> with status <0>
ip:10.44.3.39: DEVICE1: Is <Paused> in state <clk_sync_stage1> with status <0>
ip:10.44.3.53: DEVICE0: Is <Paused> in state <clk_sync_stage2> with status <0>
ip:10.44.3.39: DEVICE1: Is <Paused> in state <clk_sync_stage2> with status <0>
ip:10.44.3.53: DEVICE0: Is <Paused> in state <clk_sync_stage3> with status <0>
ip:10.44.3.39: DEVICE1: Is <Paused> in state <clk_sync_stage3> with status <0>
ip:10.44.3.53: DEVICE0: Is <Paused> in state <link_setup> with status <0>
ip:10.44.3.39: DEVICE1: Is <Paused> in state <link_setup> with status <0>
ip:10.44.3.53: DEVICE0: Is <Paused> in state <opt_setup_stage1> with status <0>
ip:10.44.3.39: DEVICE1: Is <Paused> in state <opt_setup_stage1> with status <0>
ip:10.44.3.53: DEVICE0: Is <Paused> in state <opt_setup_stage2> with status <0>
ip:10.44.3.39: DEVICE1: Is <Paused> in state <opt_setup_stage2> with status <0>
ip:10.44.3.53: DEVICE0: Is <Paused> in state <opt_setup_stage3> with status <0>
ip:10.44.3.39: DEVICE1: Is <Paused> in state <opt_setup_stage3> with status <0>
ip:10.44.3.53: DEVICE0: Is <Paused> in state <opt_setup_stage4> with status <0>
ip:10.44.3.39: DEVICE1: Is <Paused> in state <opt_setup_stage4> with status <0>
ip:10.44.3.53: DEVICE0: Is <Paused> in state <opt_setup_stage5> with status <0>
ip:10.44.3.39: DEVICE1: Is <Paused> in state <opt_setup_stage5> with status <0>
ip:10.44.3.53: DEVICE0: Is <Paused> in state <clocks_enable> with status <0>
ip:10.44.3.39: DEVICE1: Is <Paused> in state <clocks_enable> with status <0>
ip:10.44.3.53: DEVICE0: Is <Paused> in state <link_enable> with status <0>
ip:10.44.3.39: DEVICE1: Is <Paused> in state <link_enable> with status <0>
HMC7044s CAP bank select:  [14, 14, 14, 13, 13, 14, 13]
JESD Link status: DATA (84a50000.axi-jesd204-rx)
JESD Link status: DATA (84a30000.axi-jesd204-tx)
JESD Link status: DATA (84a70000.axi-jesd204-rx)
JESD Link status: DATA (84a50000.axi-jesd204-rx)
JESD Link status: DATA (84a30000.axi-jesd204-tx)
JESD Link status: DATA (84a70000.axi-jesd204-rx)
JESD SYSREF captured: Yes (84a50000.axi-jesd204-rx)
JESD SYSREF captured: Yes (84a30000.axi-jesd204-tx)
JESD SYSREF captured: Yes (84a70000.axi-jesd204-rx)
JESD SYSREF captured: Yes (84a50000.axi-jesd204-rx)
JESD SYSREF captured: Yes (84a30000.axi-jesd204-tx)
JESD SYSREF captured: Yes (84a70000.axi-jesd204-rx)
JESD SYSREF alignment error: No (84a50000.axi-jesd204-rx)
JESD SYSREF alignment error: No (84a30000.axi-jesd204-tx)
JESD SYSREF alignment error: No (84a70000.axi-jesd204-rx)
JESD SYSREF alignment error: No (84a50000.axi-jesd204-rx)
JESD SYSREF alignment error: No (84a30000.axi-jesd204-tx)
JESD SYSREF alignment error: No (84a70000.axi-jesd204-rx)
JESD 84a50000.axi-jesd204-rx:  0 0 0 0 0 0 0 0
JESD 84a70000.axi-jesd204-rx:  0 0 0 0 0 0 0 0
JESD 84a50000.axi-jesd204-rx:  0 0 0 0 0 0 0 0
JESD 84a70000.axi-jesd204-rx:  0 0 0 0 0 0 0 0
JESD 84a50000.axi-jesd204-rx:  Yes Yes Yes Yes Yes Yes Yes Yes
JESD 84a70000.axi-jesd204-rx:  Yes Yes Yes Yes Yes Yes Yes Yes
JESD 84a50000.axi-jesd204-rx:  Yes Yes Yes Yes Yes Yes Yes Yes
JESD 84a70000.axi-jesd204-rx:  Yes Yes Yes Yes Yes Yes Yes Yes
JESD 84a50000.axi-jesd204-rx:  Yes Yes Yes Yes Yes Yes Yes Yes
JESD 84a70000.axi-jesd204-rx:  Yes Yes Yes Yes Yes Yes Yes Yes
JESD 84a50000.axi-jesd204-rx:  Yes Yes Yes Yes Yes Yes Yes Yes
JESD 84a70000.axi-jesd204-rx:  Yes Yes Yes Yes Yes Yes Yes Yes
###########
Across Chip (A):	 16.590096
Across FMC8 (A):	 173.989377
Across Chip (B):	 7.867138
Across FMC8 (B):	 -164.080114
Across SoM (AB):	 39.479836
###########

Troubleshooting

On the primary setup check the status of the external clockchip. This chip reporting status Unsynchronized is expected, since this chip is only frequency locked against a 30.720MHz reference clock.

This specifies any shell prompt running on the target

root@analog:~# iio_attr -D hmc7044-ext status
--- PLL1 ---
Status: Locked
Using:  CLKIN1 @ 30720000 Hz
PFD:    3840 kHz
--- PLL2 ---
Status: Locked (Unsynchronized)
Frequency:      2949120000 Hz (Autocal cap bank value: 13)
SYSREF Status:  Valid & Locked
SYNC Status:    Unsynchronized
Lock Status:    PLL1 & PLL2 Locked
root@analog:~# 

On the primary and secondary setup check the status of all clockchips in the topology.

This specifies any shell prompt running on the target

root@analog:~# iio_attr -D hmc7044-car status
--- PLL1 ---
Status: Locked
Using:  CLKIN1 @ 30720000 Hz
PFD:    7680 kHz
--- PLL2 ---
Status: Locked (Synchronized)
Frequency:      2949120000 Hz (Autocal cap bank value: 14)
SYSREF Status:  Valid & Locked
SYNC Status:    Synchronized
Lock Status:    PLL1 & PLL2 Locked

root@analog:~# iio_attr -D hmc7044 status
--- PLL1 ---
Status: Locked
Using:  CLKIN1 @ 30720000 Hz
PFD:    30720 kHz
--- PLL2 ---
Status: Locked (Synchronized)
Frequency:      2949120000 Hz (Autocal cap bank value: 14)
SYSREF Status:  Valid & Locked
SYNC Status:    Synchronized
Lock Status:    PLL1 & PLL2 Locked

root@analog:~# iio_attr -D hmc7044-fmc status
--- PLL1 ---
Status: Locked
Using:  CLKIN1 @ 30720000 Hz
PFD:    30720 kHz
--- PLL2 ---
Status: Locked (Synchronized)
Frequency:      2949120000 Hz (Autocal cap bank value: 14)
SYSREF Status:  Valid & Locked
SYNC Status:    Synchronized
Lock Status:    PLL1 & PLL2 Locked
root@analog:~# 

resources/eval/user-guides/adrv9009-zu11eg/syncronization.txt · Last modified: 09 Nov 2020 11:00 by mhennerich