stream

Models for mio.stream_daq

class mio.models.stream.ADCScaling(*, ref_voltage: float = 1.1, bitdepth: int = 8, battery_div_factor: float = 5.0, vin_div_factor: float = 11.3)

Configuration for the ADC scaling factors

battery_div_factor: float
bitdepth: int
model_config = {}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

ref_voltage: float
scale_battery_voltage(voltage_raw: float) → float

Scale raw input ADC voltage to Volts

Parameters:

voltage_raw – Voltage as output by the ADC

Returns:

Scaled voltage

Return type:

float

scale_input_voltage(voltage_raw: float) → float

Scale raw input ADC voltage to Volts

Parameters:

voltage_raw – Voltage as output by the ADC

Returns:

Scaled voltage

Return type:

float

vin_div_factor: float
class mio.models.stream.StreamBufferHeader(*, linked_list: int, frame_num: int, buffer_count: int, frame_buffer_count: int, write_buffer_count: int, dropped_buffer_count: int, timestamp: int, write_timestamp: int, pixel_count: int, battery_voltage_raw: int, input_voltage_raw: int)

Refinements of BufferHeader for StreamDaq

property adc_scaling: ADCScaling | None

ADCScaling applied to voltage readings

property battery_voltage: float

Scaled battery voltage in Volts.

battery_voltage_raw: int
classmethod from_buffer(buffer: bytes, format: BufferHeaderFormat, config: StreamDevConfig) → tuple[Self, ndarray]

Split a raw buffer from a streaming device into a header and a data payload, typically a 1-dimensional pixel array from a chunk of a video frame

property input_voltage: float

Scaled input voltage in Volts.

input_voltage_raw: int
model_config = {}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

model_post_init(context: Any, /) → None

This function is meant to behave like a BaseModel method to initialize private attributes.

It takes context as an argument since that’s what pydantic-core passes when calling it.

Parameters:

  • self – The BaseModel instance.

  • context – The context.

pixel_count: int
class mio.models.stream.StreamBufferHeaderFormat(*, id: Annotated[str, _PydanticGeneralMetadata(pattern='[\\w\\-\\/#]+')], mio_model: Annotated[str, AfterValidator(func=_is_identifier)] = None, mio_version: str = '0.7.1.dev89+g099d1d4', linked_list: int, frame_num: int, buffer_count: int, frame_buffer_count: int, write_buffer_count: int, dropped_buffer_count: int, timestamp: int, write_timestamp: int, pixel_count: int, battery_voltage_raw: int, input_voltage_raw: int)

Refinements of BufferHeaderFormat for StreamDaq

Parameters:

  • pixel_count (int) – Number of pixels in the buffer.

  • battery_voltage (int) – Battery voltage. This is currently raw ADC value. Mapping to mV will be documented in device documentation.

  • vin_voltage (int) – Input voltage. This is currently raw ADC value. Mapping to mV will be documented in device documentation.

battery_voltage_raw: int
input_voltage_raw: int
model_config = {}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

pixel_count: int
class mio.models.stream.StreamDevConfig(*, id: Annotated[str, _PydanticGeneralMetadata(pattern='[\\w\\-\\/#]+')], mio_model: Annotated[str, AfterValidator(func=_is_identifier)] = None, mio_version: str = '0.7.1.dev89+g099d1d4', device: Literal['OK', 'UART'], bitstream: Path | None = None, port: str | None = None, baudrate: int | None = None, frame_width: int, frame_height: int, fs: int = 20, preamble: bytes, header_len: int, pix_depth: int = 8, buffer_block_length: int, block_size: int, num_buffers: int, reverse_header_bits: bool = False, reverse_header_bytes: bool = False, reverse_payload_bits: bool = False, reverse_payload_bytes: bool = False, dummy_words: int = 0, adc_scale: ADCScaling | None = ADCScaling(ref_voltage=1.1, bitdepth=8, battery_div_factor=5.0, vin_div_factor=11.3), runtime: StreamDevRuntime = StreamDevRuntime(serial_buffer_queue_size=10, frame_buffer_queue_size=5, image_buffer_queue_size=5, queue_put_timeout=5, plot=StreamPlotterConfig(keys=['timestamp', 'buffer_count', 'frame_buffer_count'], update_ms=1000, history=500), csvwriter=CSVWriterConfig(buffer=100)))

Format model used to parse DAQ configuration yaml file (examples are in ./config) The model attributes are key-value pairs needed for reconstructing frames from data streams.

Parameters:

  • device (str) – Interface hardware used for receiving data. Current options are “OK” (Opal Kelly XEM 7310) and “UART” (generic UART-USB converters). Only “OK” is supported at the moment.

  • bitstream (str, optional) – Required when device is “OK”. The configuration bitstream file to upload to the Opal Kelly board. This uploads a Manchester decoder HDL and different bitstream files are required to configure different data rates and bit polarity. This is a binary file synthesized using Vivado, and details for generating this file will be provided in later updates.

  • port (str, optional) – Required when device is “UART”. COM port connected to the UART-USB converter.

  • baudrate (Optional[int]) – Required when device is “UART”. Baudrate of the connection to the UART-USB converter.

  • frame_width (int) – Frame width of transferred image. This is used to reconstruct image.

  • frame_height (int) – Frame height of transferred image. This is used to reconstruct image.

  • fs (int) – Framerate of acquired stream

  • preamble (str) – 32-bit preamble used to locate the start of each buffer. The header and image data follows this preamble. This is used as a hex but imported as a string because yaml doesn’t support hex format.

  • header_len (int, optional) – Length of header in bits. (For 32-bit words, 32 * number of words) This is useful when not all the variable/words in the header are defined in MetadataHeaderFormat. The user is responsible to ensure that header_len is larger than the largest bit position defined in MetadataHeaderFormat otherwise unexpected behavior might occur.

  • pix_depth (int, optional) – Bit-depth of each pixel, by default 8.

  • buffer_block_length (int) – Defines the data buffer structure. This value needs to match the Miniscope firmware. Number of blocks per each data buffer. This is required to calculate the number of pixels contained in one data buffer.

  • block_size (int) – Defines the data buffer structure. This value needs to match the Miniscope firmware. Number of 32-bit words per data block. This is required to calculate the number of pixels contained in one data buffer.

  • num_buffers (int) – Defines the data buffer structure. This value needs to match the Miniscope firmware. This is the number of buffers that the source microcontroller cycles around. This isn’t strictly required for data reconstruction but useful for debugging.

  • reverse_header_bits (bool, optional) – If True, reverse the bits within each byte of the header. Default is False.

  • reverse_header_bytes (bool, optional) – If True, reverse the byte order within each 32-bit word of the header. This is used for handling endianness in systems where the byte order needs to be swapped. Default is False.

  • reverse_payload_bits (bool, optional) – If True, reverse the bits within each byte of the payload. Default is False.

  • reverse_payload_bytes (bool, optional) – If True, reverse the byte order within each 32-bit word of the payload. This is used for handling endianness in systems where the byte order needs to be swapped. Default is False.

  • dummy_words (int, optional) – Number of 32-bit dummy words in the header. This is used to stabilize clock recovery in FPGA Manchester decoder. This value does not have a meaning for image recovery.

  • ..todo:: – Move port (for USART) to a user config area. This should make this pure device config.

adc_scale: ADCScaling | None
baudrate: int | None
bitstream: Path | None
block_size: int
buffer_block_length: int
device: Literal['OK', 'UART']
dummy_words: int
classmethod ensure_exists(value: Path | None) → Path | None

If a bitstream file has been provided, ensure it exists

frame_height: int
frame_width: int
fs: int
header_len: int
model_config = {}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

model_post_init(context: Any, /) → None

This function is meant to behave like a BaseModel method to initialize private attributes.

It takes context as an argument since that’s what pydantic-core passes when calling it.

Parameters:

  • self – The BaseModel instance.

  • context – The context.

num_buffers: int
pix_depth: int
port: str | None
preamble: bytes
classmethod preamble_to_bytes(value: str | bytes | int) → bytes

Cast preamble to bytes.

Parameters:

value (str, bytes, int) – Recast from str (in yaml like preamble: "0x12345" ) or int (in yaml like preamble: 0x12345

Returns:

bytes

property px_per_buffer: int

Number of pixels per buffer

classmethod resolve_relative(value: Path) → Path

If we are given a relative path to a bitstream, resolve it relative to the device path

reverse_header_bits: bool
reverse_header_bytes: bool
reverse_payload_bits: bool
reverse_payload_bytes: bool
runtime: StreamDevRuntime
class mio.models.stream.StreamDevRuntime(*, serial_buffer_queue_size: int = 10, frame_buffer_queue_size: int = 5, image_buffer_queue_size: int = 5, queue_put_timeout: int = 5, plot: StreamPlotterConfig | None = StreamPlotterConfig(keys=['timestamp', 'buffer_count', 'frame_buffer_count'], update_ms=1000, history=500), csvwriter: CSVWriterConfig | None = CSVWriterConfig(buffer=100))

Runtime configuration for StreamDaq

Included within StreamDevConfig to separate config that is not unique to the device, but how that device is controlled at runtime.

csvwriter: CSVWriterConfig | None
frame_buffer_queue_size: int
image_buffer_queue_size: int
model_config = {}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

plot: StreamPlotterConfig | None
queue_put_timeout: int
serial_buffer_queue_size: int