pyIDI source code

pyIDI base class

class pyidi.pyidi.pyIDI(input_file, root=None)

Bases: object

The pyIDI base class represents the video to be analysed.

__init__(input_file, root=None)

Constructor of the pyIDI class.

Parameters:

• input_file (str or np.ndarray) – the video file to be analysed. Can be a name of the cih/cihx file, path to images directory, video file, or a 3D numpy array.

• root (str) – root directory of the video file. Only used when the input file is a np.ndarray. Defaults to None.

set_method(method, **kwargs)

Set displacement identification method on video. To configure the method, use method.configure()

Available methods: — [Available method names and descriptions go here.] —

Parameters:

method (IDIMethod or str) – the method to be used for displacement identification.

set_points(points=None, method=None, **kwargs)

Set points that will be used to calculate displacements. If points is None and a method has aready been set on this pyIDI instance, the method object’s get_point is used to get method-appropriate points.

show_points(**kwargs)

Show selected points on image.

show_field(field, scale=1.0, width=0.5)

Show displacement field on image.

Parameters:

• field (ndarray) – Field of displacements (number_of_points, 2)

• scale – scale the field, defaults to 1.

• scale – float, optional

• width – width of the arrow, defaults to 0.5

• width – float, optional

get_displacements(autosave=True, **kwargs)

Calculate the displacements based on chosen method.

Method docstring: — Method is not set. Please use the set_method method. —

Displacement identification methods

IDIMetod base class

class pyidi.methods.idi_method.IDIMethod(video, *args, **kwargs)

Bases: object

Common functions for all methods.

__init__(video, *args, **kwargs)

The image displacement identification method constructor.

For more configuration options, see method.configure()

configure(*args, **kwargs)

Configure the displacement identification method here.

calculate_displacements(video, *args, **kwargs)

Calculate the displacements of set points here. The result should be saved into the self.displacements attribute.

Simplified optical flow

class pyidi.methods._simplified_optical_flow.SimplifiedOpticalFlow(video, *args, **kwargs)

Bases: IDIMethod

Displacmenet computation based on Simplified Optical Flow method [1].

Literature:

[1] Javh, J., Slavič, J., & Boltežar, M. (2017). The subpixel resolution

of optical-flow-based modal analysis. Mechanical Systems and Signal Processing, 88, 89–99.

[2] Lucas, B. D., & Kanade, T. (1981). An Iterative Image Registration

Technique with an Application to Stereo Vision. In Proceedings of the 7th International Joint Conference on Artificial Intelligence - Volume 2 (pp. 674–679). San Francisco, CA, USA: Morgan Kaufmann Publishers Inc.

configure(subset_size=3, pixel_shift=False, convert_from_px=1.0, mraw_range='all', mean_n_neighbours=0, zero_shift=False, progress_bar=True, reference_range=(0, 100))

Set the attributes, compute reference image and gradients.

Parameters:

• video (object) – ‘parent’ object

• subset_size – size of the averaging subset, defaults to 3

• subset_size – int, optional

• pixel_shift – use pixel shift or not?, defaults to False

• pixel_shift – bool, optional

• convert_from_px – distance unit per pixel, defaults to 1.

• convert_from_px – float or int, optional

• mraw_range – what range of images to calculate into displacements, defaults to ‘all’

• mraw_range – str or tuple, optional

• mean_n_neighbours – average the displacements of neighbouring points (how many points), defaults to 0

• mean_n_neighbours – int, optional

• zero_shift – shift the mean of the signal to zero?, defaults to False

• zero_shift – bool, optional

• progress_bar – show progress bar while calculating the displacements, defaults to True

• progress_bar – bool, optional

• reference_range – what range of images is averaged into reference image, defaults to (0, 100)

• reference_range – tuple, optional

calculate_displacements(video)

Calculate the displacements of set points here. The result should be saved into the self.displacements attribute.

displacement_averaging()

Calculate the average of displacements.

pixel_shift_fun(i, points, image_shape)

Pixel shifting implementation. Points that are going outside of the image range are excluded.

_reference()

Calculation of the reference image, image gradients and gradient amplitudes.

Parameters:

• images – Images to average. Usually the first 100 images.

• subset_size – Size of the subset to average.

Returns:

Reference image, image gradient in 0 direction, image gradient in 1 direction, gradient magnitude

subset(data, subset_size)

Calculating a filtered image.

Calculates a filtered image with subset of d. It sums the area of d x d.

Parameters:

• data – Image that is to be filtered.

• subset_size – Size of the subset.

Returns:

Filtered image.

static get_points(video, **kwargs)

Determine the points.

class pyidi.methods._simplified_optical_flow.PickPoints(video, subset, axis, min_grad)

Bases: object

Pick the area of interest.

Select the points with highest gradient in vertical direction.

__init__(video, subset, axis, min_grad)

inside_polygon(x, y, points)

Return True if a coordinate (x, y) is inside a polygon defined by a list of verticies [(x1, y1), (x2, x2), … , (xN, yN)].

Reference: http://www.ariel.com.au/a/python-point-int-poly.html

The Lucas-Kanade algorithm for translations

class pyidi.methods._lucas_kanade.LucasKanade(video, *args, **kwargs)

Bases: IDIMethod

Translation identification based on the Lucas-Kanade method using least-squares iterative optimization with the Zero Normalized Cross Correlation optimization criterium.

configure(roi_size=(9, 9), pad=2, max_nfev=20, tol=1e-08, int_order=3, verbose=1, show_pbar=True, processes=1, pbar_type='atpbar', multi_type='mantichora', resume_analysis=True, process_number=0, reference_image=0, mraw_range='full', use_numba=False)

Displacement identification based on Lucas-Kanade method, using iterative least squares optimization of translatory transformation parameters to determine image ROI translations.

Parameters:

• video (object) – parent object

• roi_size (tuple, list, optional) – (h, w) height and width of the region of interest. ROI dimensions should be odd numbers. Defaults to (9, 9)

• pad (int, optional) – size of padding around the region of interest in px, defaults to 2

• max_nfev (int, optional) – maximum number of iterations in least-squares optimization, defaults to 20

• tol (float, optional) – tolerance for termination of the iterative optimization loop. The minimum value of the optimization parameter vector norm.

• int_order (int, optional) – interpolation spline order

• verbose (int, optional) – show text while running, defaults to 1

• show_pbar (bool, optional) – show progress bar, defaults to True

• processes (int, optional, defaults to 1.) – number of processes to run

• pbar_type (str, optional) – type of the progress bar (‘tqdm’ or ‘atpbar’), defaults to ‘atpbar’

• multi_type (str, optional) – type of multiprocessing used (‘multiprocessing’ or ‘mantichora’), defaults to ‘mantichora’

• resume_analysis – if True, the last analysis results are loaded and computation continues from last computed time point.

• process_number (int, optional) – User should not change this (for multiprocessing purposes - to indicate the process number)

• reference_image (int or tuple or ndarray) – The reference image for computation. Can be index of a frame, tuple (slice) or numpy.ndarray that is taken as a reference.

• mraw_range (tuple or "full") – Part of the video to process. If “full”, a full video is processed. If first element of tuple is not 0, a appropriate reference image should be chosen.

• use_numba (bool) – Use numba.njit for computation speedup. Currently not implemented.

_set_mraw_range()

Set the range of the video to be processed.

calculate_displacements(video, **kwargs)

Calculate displacements for set points and roi size.

kwargs are passed to configure method. Pre-set arguments (using configure) are NOT changed!

optimize_translations(G, F_spline, maxiter, tol, d_subpixel_init=(0, 0))

Determine the optimal translation parameters to align the current image subset G with the interpolated reference image subset F.

Parameters:

• G (array of shape roi_size) – the current image subset.

• F_spline (scipy.interpolate.RectBivariateSpline) – interpolated referencee image subset

• maxiter (int) – maximum number of iterations

• tol (float) – convergence criterium

• d_subpixel_init – initial subpixel displacement guess, relative to the integrer position of the image subset G

Returns:

the obtimal subpixel translation parameters of the current image, relative to the position of input subset G.

Return type:

array of size 2

_padded_slice(point, roi_size, image_shape, pad=None)

Returns a slice that crops an image around a given point center, roi_size and pad size. If the resulting slice would be out of bounds of the image to be sliced (given by image_shape), the slice is snifted to be on the image edge and a warning is issued.

Parameters:

• point (array_like of size 2, (y, x)) – The center point coordiante of the desired ROI.

• roi_size – Size of desired cropped image (y, x). type roi_size: array_like of size 2, (h, w)

• image_shape – Shape of the image to be sliced, (h, w). type image_shape: array_like of size 2, (h, w)

• pad (int, optional, defaults to None) – Pad border size in pixels. If None, the video.pad attribute is read.

Return crop_slice:

tuple (yslice, xslice) to use for image slicing.

_pbar_range(*args, **kwargs)

Set progress bar range or normal range.

_set_reference_image(video, reference_image)

Set the reference image.

_interpolate_reference(video)

Interpolate the reference image.

Each ROI is interpolated in advanced to save computation costs. Meshgrid for every ROI (without padding) is also determined here and is later called in every time iteration for every point.

Parameters:

video (object) – parent object

property roi_size

roi_size attribute getter

show_points(video, figsize=(15, 5), cmap='gray', color='r')

Shoe points to be analyzed, together with ROI borders.

Parameters:

• figsize – matplotlib figure size, defaults to (15, 5)

• cmap – matplotlib colormap, defaults to ‘gray’

• color – marker and border color, defaults to ‘r’

create_temp_files(init_multi=False)

Temporary files to track the solving process.

This is done in case some error occures. In this eventuality the calculation can be resumed from the last computed time point.

Parameters:

init_multi (bool, optional) – when initialization multiprocessing, defaults to False

clear_temp_files()

Clearing the temporary files.

update_log(last_time)

Updating the log file.

A new last time is written in the log file in order to track the solution process.

Parameters:

last_time (int) – Last computed time point (index)

resume_temp_files()

Reload the settings written in the temporary files.

When resuming the computation of displacement, the settings are loaded from the previously created temporary files.

temp_files_check()

Checking the settings of computation.

The computation can only be resumed if all the settings and data are the same as with the original analysis. This function checks that (writing all the setting to dict and comparing the json dump of the dicts).

If the settings are the same but the points are not, a new analysis is also started. To set the same points, check the temp_pyidi folder.

Returns:

Whether to resume analysis or not

Return type:

bool

create_settings_dict()

Make a dictionary of the chosen settings.

_make_comparison_dict()

Make a dictionary for comparing the original settings with the current settings.

Used for finding out if the analysis should be resumed or not.

Returns:

Settings

Return type:

dict

pyidi.methods._lucas_kanade.multi(video, processes)

Splitting the points to multiple processes and creating a pool of workers.

Parameters:

• video (object) – the video object with defined attributes

• processes (int) – number of processes. If negative, the number of processes is set to psutil.cpu_count + processes.

Returns:

displacements

Return type:

ndarray

pyidi.methods._lucas_kanade.worker(points, idi_kwargs, method_kwargs, i)

A function that is called when for each job in multiprocessing.

Postprocessing

pyidi.postprocessing._motion_magnification.mode_shape_magnification(displacements: ndarray, magnification_factor: int | float, video: pyidi.pyIDI | None = None, image: ndarray | memmap | None = None, points: ndarray | None = None, background_brightness: float = 0.3, show_undeformed: bool = False) → ndarray

Create an image of a magnified mode-shape of a structure. If a ‘pyidi.pyIDI’ class instance is input as argument ‘video’, the argument ‘image’ is set to ‘video.mraw[0]’ and the argument ‘points’ is set to ‘video.points’. These values can be overwritten by specifying the ‘image’ and ‘points’ arguments explicitly.

Parameters:

• displacements (numpy.ndarray) – displacement (mode-shape) vector

• magnification_factor (int or float) – magnification factor

• video (pyidi.pyIDI or None, optional) – pyIDI class instance, defaults to None

• image (numpy.ndarray, numpy.memmap or None, optional) – the reference image, on which mode-shape magnification is performed, defaults to None

• points (numpy.ndarray or None, optional) – image coordinates, where displacements ‘displacements’ are defined, defaults to None

• background_brightness – brightness of the background, expected values in range [0, 1], defaults to 0.3

• show_undeformed (bool, optional) – Show the reference image (argument ‘image’) underneath the magnified mode-shape, defaults to False

Returns:

image of a magnified mode-shape of the structure

Return type:

numpy.ndarray

pyidi.postprocessing._motion_magnification.animate(displacements: ndarray, magnification_factor: int | float, video: pyidi.pyIDI | None = None, image: ndarray | memmap | None = None, points: ndarray | None = None, fps: int = 30, n_periods: int = 3, filename: str = 'mode_shape_mag_video', output_format: str = 'gif', background_brightness: float = 0.3, show_undeformed: bool = False) → None

Create a video of a magnified mode-shape of a structure. If a ‘pyidi.pyIDI’ class instance is input as argument ‘video’, the argument ‘image’ is set to ‘video.mraw[0]’ and the argument ‘points’ is set to ‘video.points’. These values can be overwritten by specifying the ‘image’ and ‘points’ arguments explicitly.

Parameters:

• displacements (numpy.ndarray) – displacement vector

• magnification_factor (int or float) – magnification factor

• video (pyidi.pyIDI or None, optional) – pyIDI class instance, defaults to None

• image (numpy.ndarray, numpy.memmap or None, optional) – the reference image, on which mode-shape magnification is performed, defaults to None

• points (numpy.ndarray or None, optional) – image coordinates, where displacements ‘displacements’ are defined, defaults to None

• fps (int, optional) – framerate of the created video, defaults to 30

• n_periods (int, optional) – number of periods of oscilation to be animated, defaults to 3

• filename (str) – the name of the output video file defaults to ‘mode_shape_mag_video’

• output_format (str, optional) – output format of the video, selected from ‘gif’, ‘mp4’, ‘avi’, ‘mov’, defaults to ‘gif’

• background_brightness – brightness of the background, expected values in range [0, 1], defaults to 0.3

• show_undeformed (bool, optional) – Show the reference image (argument ‘image’) underneath the magnified mode-shape, defaults to True

pyidi.postprocessing._motion_magnification.create_mesh(points, disp, mag_fact)

Generates a planar mesh of triangles based on the input set of points. Then generates the deformed planar mesh of triangles based on the displacement vectors ‘disp’, scaled by the magnification factor ‘mag_fact’.

pyidi.postprocessing._motion_magnification.init_output_image(input_image, mesh, mesh_def, bb, bu)

Initialze the output image. The output image needs to be large enough to prevent clipping of the motion magnified shape.

pyidi.postprocessing._motion_magnification.warp_image_elements(img_in, img_out, mesh, mesh_def, a, b)

Warp image elements based on mesh and deformed mesh nodes.

Video Reader

Module for reading video files from high-speed video recordings.

@author: Ivan Tomac (itomac@fesb.hr), Klemen Zaletelj (klemen.zaletelj@fs.uni-lj.si), Janko Slavič (janko.slavic@fs.uni-lj.si)

class pyidi.video_reader.VideoReader(input_file, root=None)

Bases: object

Manages reading of high-speed video recordings. The video recording can be any of the supported file formats which includes image streams, video files or memory map for “mraw” file format.

This applies to frames from image and video file formats: Reader returns the frame as a monochrome image. For colour images the “Y” (luma) is default channel, but other channels can be selected (“R”, “G”, “B”, “Y”). The reader returns image in 2D “numpy.array” (“height, width”) of type “numpy.uint8” or “numpy.uint16” depending on the bit depth of the image file, e.g. 12 bit depth images are returned as “numpy.array” of type “numpy.uint16”.

__init__(input_file, root=None)

The video recording is initialized by providing the path to the image/video file, “cih(x)” file from Photron, or “numpy.ndarray”. For image stream it is enough to provide the path to the any image file in the sequence. Images in stream must be in the same directory and named in the way that can be sorted in the correct order, e.g. for stream of 10000 images file names should be: “im_0000.ext, …, im_9999.ext”. Image formats that support multiple images, such as “gif”, “tif” are supported too. Upgrade is needed to enable higher bit depth then 8 bit for video file formats.

Parameters:

• input_file (str) – path to the image/video or “cih(x)” file

• root (str) – root directory of the image/video file. Only used when the input file is a “np.ndarray”. Defaults to None.

get_frame(frame_number, *args)

Returns the “frame_number”-th frame from the video. Frames from image and video files are checked for the bit depth and converted to 8 or 16 bit depth if needed. The frames from “numpy.ndarray” and “mraw” files are returned as they are.

Parameters:

• frame_number (int) – frame number

• args – additional arguments to be passed to the image readers to handle multiple channels in image

Returns:

image (monochrome)

_get_frame_from_image(frame_number, use_channel='Y')

Reads the frame from the image stream, or image file containing multiple images. Colour images are assumed to be in “RGB(A)” format and they are automatically converted to “YUV” and “Y”(luma) channel is used. The ‘use_channel’ parameter can be used to select other the channels. The supported channels are R, G, B, Y (luma).

Parameters:

• frame_number (int) – frame number

• use_channel – “R”, “G”, “B”, “Y” (luma), defaults to “Y”

Returns:

image (monochrome)

_get_frame_from_video_file(frame_number, use_channel='Y')

Reads the frame from the video file which is supported by the “imagio.v3” “pyav” plug-in.

Parameters:

• frame_number (int) – frame number

• use_channel – “R”, “G”, “B”, “Y” (luma), defaults to “Y”

Returns:

monochrome image in 8 bit depth (note: needs upgrade to support higher bit depth)

pyidi.video_reader._rgb2luma(rgb_image)

Converts “RGB” image to “YUV” and returns only “Y” (luma) component.

Parameters:

rgb_image (numpy.array) – “RGB” image “(w, h, channels)”

Returns:

luma image