Stitching Images (minimal example for small data)
[1]:
import cv2
import matplotlib.pyplot as plt
import numpy as np
import microscopy_data_analysis as mda
[2]:
#for the next line to work, the file create_example_for_grid_stitching.py
#should be in the same directory as this notebook
import create_data_for_grid_stitching
create_data_for_grid_stitching.run_script()
[3]:
# get filepaths to images
example_path="example_images"
pathlist=mda.get_files_of_format(example_path,"tif")
N=len(pathlist)
pathlist[:3]
[3]:
['example_images/image_00.tif',
'example_images/image_01.tif',
'example_images/image_02.tif']
[4]:
# load images into RAM (recommended only for small dataset)
img_list=[]
for imgpath in pathlist:
img_list.append(cv2.imread(imgpath,0))
img_list[0].shape
[4]:
(500, 500)
[5]:
%matplotlib inline
# create and plot some estimated positions for stitching the images
positions=np.zeros([N,2])
counter=0
for i in range(5):
for j in range(5):
# assuming 25% overlap with an image width of 500 pixels,
# the next image should start after 375 pixels
positions[counter]=j*375,5*375-i*375
counter+=1
plt.figure(figsize=[5,5])
plt.plot(positions[:,0],positions[:,1],'x')
for i in range(len(positions)):
plt.text(positions[i,0],positions[i,1],str(i))
plt.xlabel(r"x [mm]")
plt.ylabel(r"y [mm]")
plt.axis("equal")
plt.show()
The mode ‘memory’ is only recommended for a small datasets as everything is stored in RAM. 25 low resolution 500x500 images with low bit-depth uint8 would correspond to ~6.25Mb. Copies in float32 will be needed, inflating the data by a factor of 4, which still is tiny.
But for example 400 images with resolution 2048x2048 in float32 would take ~6.25Gb of RAM. In that case (see other example), considering that also copies are needed for processing, the mode ‘storage’ is recommended (assuming ~16Gb of RAM).
Rule of thumb: If the size of all images in float32 times 4 is smaller than the amount of RAM, then the mode “memory” is fine.
[6]:
stack=mda.image_stack(mode='memory')
please provide the images, as a list of images via 'set_img_list'
[7]:
stack.set_img_list(img_list)
[8]:
stack.sniff_dimensions()
# check values
stack.dimensions[:3]
[8]:
array([[500, 500],
[500, 500],
[500, 500]])
[9]:
# since our estimated position are given in pixels,
# the correct value would be 1,
# but let's assume the value is not precise
stack.set_units_per_pixel(1.1)
[10]:
stack.set_positions(positions)
polygons,anchor_points=stack.make_polygons()
[11]:
%matplotlib inline
# check for two images, if there is overlap
# also origin (anchor) of the pixel coordinates, shown as blue dot, should be
# in the left upper corner of an image
index=0
plt.plot(np.array(polygons[index].exterior.xy[0]),np.array(polygons[index].exterior.xy[1]),'k-',alpha=0.4)
plt.plot(anchor_points[index][0],anchor_points[index][1],'o')
index=1
plt.plot(np.array(polygons[index].exterior.xy[0]),np.array(polygons[index].exterior.xy[1]),'k-',alpha=0.4)
plt.axis("equal")
plt.show()
[12]:
# show all images (abstracted as polygons)
for index in range(len(stack.polygons)):
plt.plot(np.array(stack.polygons[index].exterior.xy[0]),np.array(stack.polygons[index].exterior.xy[1]),'-',alpha=0.6)
plt.axis("equal")
plt.show()
[13]:
connected_groups=stack.connection_groups()
stack.plot_connection_network(figsize=[15,15],relative=False)
[14]:
pair_shifts=stack.check_pairs()
100%|██████████| 72/72 [00:03<00:00, 19.89it/s]
[15]:
moved_polygons=stack.optimize_positions()
Iteration Total nfev Cost Cost reduction Step norm Optimality
0 1 1.0092e+05 2.00e+02
1 3 5.2491e+04 4.84e+04 1.71e+02 1.04e+02
2 5 4.0763e+04 1.17e+04 8.56e+01 4.60e+01
3 6 3.6472e+04 4.29e+03 8.56e+01 1.08e+02
4 7 3.6394e+04 7.70e+01 8.56e+01 8.52e+01
5 8 3.2557e+04 3.84e+03 2.14e+01 5.04e+01
6 9 3.1175e+04 1.38e+03 4.28e+01 4.34e+01
7 11 3.0381e+04 7.93e+02 1.07e+01 1.97e+01
8 12 2.9967e+04 4.15e+02 2.14e+01 1.87e+01
9 13 2.9817e+04 1.50e+02 2.14e+01 2.74e+01
10 14 2.9549e+04 2.68e+02 5.35e+00 1.57e+01
11 15 2.9375e+04 1.74e+02 1.07e+01 9.39e+00
12 16 2.9272e+04 1.03e+02 1.07e+01 1.21e+01
13 17 2.9206e+04 6.58e+01 1.07e+01 1.16e+01
14 18 2.9140e+04 6.69e+01 2.67e+00 6.33e+00
15 19 2.9089e+04 5.10e+01 5.35e+00 5.07e+00
16 20 2.9052e+04 3.70e+01 5.35e+00 5.16e+00
17 21 2.9027e+04 2.42e+01 5.35e+00 6.77e+00
18 22 2.9012e+04 1.56e+01 5.35e+00 6.18e+00
19 23 2.8995e+04 1.71e+01 1.34e+00 3.13e+00
20 24 2.8982e+04 1.28e+01 2.67e+00 2.82e+00
21 25 2.8973e+04 9.24e+00 2.67e+00 2.51e+00
22 26 2.8967e+04 6.11e+00 2.67e+00 3.84e+00
23 27 2.8963e+04 3.83e+00 2.67e+00 3.65e+00
24 28 2.8958e+04 4.39e+00 6.69e-01 1.70e+00
25 29 2.8955e+04 3.30e+00 1.34e+00 1.54e+00
26 30 2.8953e+04 2.39e+00 1.34e+00 1.42e+00
27 31 2.8951e+04 1.58e+00 1.34e+00 2.06e+00
28 32 2.8950e+04 9.88e-01 1.34e+00 1.94e+00
29 33 2.8949e+04 1.14e+00 3.34e-01 9.19e-01
30 34 2.8948e+04 8.71e-01 6.69e-01 7.88e-01
31 35 2.8947e+04 6.48e-01 6.69e-01 6.91e-01
32 36 2.8947e+04 4.31e-01 6.69e-01 1.02e+00
33 37 2.8947e+04 2.77e-01 6.69e-01 9.22e-01
34 38 2.8946e+04 2.92e-01 1.67e-01 4.34e-01
35 39 2.8946e+04 2.35e-01 3.34e-01 3.77e-01
36 40 2.8946e+04 1.85e-01 3.34e-01 3.12e-01
37 41 2.8946e+04 1.24e-01 3.34e-01 4.73e-01
38 42 2.8946e+04 8.12e-02 3.34e-01 4.41e-01
39 43 2.8946e+04 7.39e-02 8.36e-02 2.20e-01
40 44 2.8946e+04 6.45e-02 1.67e-01 1.76e-01
41 45 2.8946e+04 5.36e-02 1.67e-01 1.47e-01
42 46 2.8946e+04 3.72e-02 1.67e-01 2.16e-01
43 47 2.8946e+04 2.42e-02 1.67e-01 2.22e-01
44 48 2.8945e+04 2.05e-02 1.67e-01 2.36e-01
45 49 2.8945e+04 1.76e-02 4.18e-02 1.26e-01
46 50 2.8945e+04 1.39e-02 8.36e-02 7.75e-02
47 51 2.8945e+04 1.07e-02 8.36e-02 1.02e-01
48 52 2.8945e+04 7.40e-03 8.36e-02 1.06e-01
49 53 2.8945e+04 5.69e-03 8.36e-02 1.16e-01
50 54 2.8945e+04 3.48e-03 8.36e-02 1.22e-01
51 55 2.8945e+04 4.65e-03 2.09e-02 6.23e-02
52 56 2.8945e+04 3.27e-03 4.18e-02 4.57e-02
53 57 2.8945e+04 2.35e-03 4.18e-02 4.86e-02
54 58 2.8945e+04 1.62e-03 4.18e-02 5.67e-02
55 59 2.8945e+04 1.08e-03 4.18e-02 5.97e-02
56 60 2.8945e+04 1.16e-03 1.04e-02 3.02e-02
57 61 2.8945e+04 9.08e-04 2.09e-02 2.20e-02
58 62 2.8945e+04 7.11e-04 2.09e-02 2.19e-02
59 63 2.8945e+04 4.83e-04 2.09e-02 2.73e-02
60 64 2.8945e+04 3.22e-04 2.09e-02 2.92e-02
61 65 2.8945e+04 2.90e-04 5.22e-03 1.46e-02
62 66 2.8945e+04 2.53e-04 1.04e-02 1.05e-02
`ftol` termination condition is satisfied.
Function evaluations 66, initial cost 1.0092e+05, final cost 2.8945e+04, first-order optimality 1.05e-02.
[16]:
%matplotlib inline
for i in range(len(moved_polygons)):
plt.plot(np.array(moved_polygons[i].exterior.xy[0]),np.array(moved_polygons[i].exterior.xy[1]))#'k-')
plt.xlabel(r"x [$\mu$m]")
plt.ylabel(r"y [$\mu$m]")
plt.axis("equal")
plt.show()
[17]:
testmap=stack.map_from_polygons(moved_polygons,blending="average")
plt.imshow(testmap)
plt.colorbar()
plt.show()
100%|██████████| 25/25 [00:00<00:00, 606.53it/s]
Optional: second iteration (start)
[ ]:
# use the new positions of image tiles as starting estimate and repeat the process
# (does not always lead to a better result, but it's a quick second try,
# if the first result is not good)
stack.refine_positions(moved_polygons=moved_polygons)
[19]:
pair_shifts=stack.check_pairs()
100%|██████████| 72/72 [00:02<00:00, 28.42it/s]
[20]:
moved_polygons=stack.optimize_positions()
Iteration Total nfev Cost Cost reduction Step norm Optimality
0 1 4.2608e+04 1.11e+02
1 3 2.1653e+04 2.10e+04 1.63e+02 5.34e+01
2 5 1.5310e+04 6.34e+03 6.94e+01 3.18e+01
3 6 1.3233e+04 2.08e+03 1.39e+02 5.20e+01
4 8 1.0921e+04 2.31e+03 3.47e+01 1.48e+01
5 9 1.0538e+04 3.83e+02 3.47e+01 1.44e+01
6 11 1.0369e+04 1.69e+02 8.67e+00 4.62e+00
7 12 1.0284e+04 8.49e+01 8.67e+00 2.95e+00
8 13 1.0169e+04 1.15e+02 1.73e+01 3.65e+00
9 15 1.0139e+04 3.04e+01 8.67e+00 4.02e+00
10 16 1.0120e+04 1.87e+01 8.67e+00 4.03e+00
11 17 1.0102e+04 1.86e+01 2.17e+00 2.04e+00
12 18 1.0088e+04 1.37e+01 4.33e+00 1.88e+00
13 19 1.0077e+04 1.05e+01 4.33e+00 1.79e+00
14 20 1.0071e+04 6.73e+00 4.33e+00 2.32e+00
15 21 1.0066e+04 4.89e+00 4.33e+00 2.34e+00
16 22 1.0061e+04 4.91e+00 1.08e+00 1.11e+00
17 23 1.0057e+04 3.71e+00 2.17e+00 1.01e+00
18 24 1.0054e+04 2.94e+00 2.17e+00 1.00e+00
19 25 1.0052e+04 2.00e+00 2.17e+00 1.18e+00
20 26 1.0051e+04 1.42e+00 2.17e+00 1.35e+00
21 27 1.0049e+04 1.28e+00 5.42e-01 6.58e-01
22 28 1.0048e+04 1.07e+00 1.08e+00 4.89e-01
23 29 1.0047e+04 9.01e-01 1.08e+00 5.26e-01
24 30 1.0047e+04 6.58e-01 1.08e+00 5.48e-01
25 31 1.0046e+04 4.52e-01 1.08e+00 7.06e-01
26 32 1.0046e+04 3.43e-01 1.08e+00 6.55e-01
27 33 1.0046e+04 3.25e-01 2.71e-01 3.41e-01
28 34 1.0045e+04 2.58e-01 5.42e-01 2.64e-01
29 35 1.0045e+04 2.11e-01 5.42e-01 2.59e-01
30 36 1.0045e+04 1.56e-01 5.42e-01 3.29e-01
31 37 1.0045e+04 1.09e-01 5.42e-01 3.18e-01
32 38 1.0045e+04 8.06e-02 5.42e-01 3.88e-01
33 39 1.0045e+04 8.28e-02 1.35e-01 1.90e-01
34 40 1.0045e+04 6.35e-02 2.71e-01 1.34e-01
35 41 1.0045e+04 5.22e-02 2.71e-01 1.49e-01
36 42 1.0045e+04 3.78e-02 2.71e-01 1.50e-01
37 43 1.0045e+04 2.63e-02 2.71e-01 1.88e-01
38 44 1.0045e+04 2.09e-02 2.71e-01 1.72e-01
39 45 1.0045e+04 2.05e-02 6.77e-02 8.40e-02
40 46 1.0045e+04 1.56e-02 1.35e-01 6.92e-02
41 47 1.0044e+04 1.26e-02 1.35e-01 6.74e-02
42 48 1.0044e+04 9.22e-03 1.35e-01 8.54e-02
43 49 1.0044e+04 6.62e-03 1.35e-01 8.18e-02
44 50 1.0044e+04 4.84e-03 1.35e-01 9.80e-02
45 51 1.0044e+04 5.21e-03 3.39e-02 4.82e-02
46 52 1.0044e+04 3.89e-03 6.77e-02 3.37e-02
47 53 1.0044e+04 3.16e-03 6.77e-02 3.79e-02
48 54 1.0044e+04 2.28e-03 6.77e-02 3.77e-02
49 55 1.0044e+04 1.59e-03 6.77e-02 4.69e-02
50 56 1.0044e+04 1.28e-03 6.77e-02 4.28e-02
51 57 1.0044e+04 1.28e-03 1.69e-02 2.06e-02
52 58 1.0044e+04 9.55e-04 3.39e-02 1.72e-02
53 59 1.0044e+04 7.70e-04 3.39e-02 1.67e-02
54 60 1.0044e+04 5.60e-04 3.39e-02 2.10e-02
55 61 1.0044e+04 4.07e-04 3.39e-02 2.00e-02
56 62 1.0044e+04 2.96e-04 3.39e-02 2.38e-02
57 63 1.0044e+04 3.25e-04 8.47e-03 1.17e-02
58 64 1.0044e+04 2.40e-04 1.69e-02 8.17e-03
59 65 1.0044e+04 1.94e-04 1.69e-02 9.25e-03
60 66 1.0044e+04 1.40e-04 1.69e-02 9.19e-03
61 67 1.0044e+04 9.83e-05 1.69e-02 1.13e-02
`ftol` termination condition is satisfied.
Function evaluations 67, initial cost 4.2608e+04, final cost 1.0044e+04, first-order optimality 1.13e-02.
[21]:
%matplotlib inline
for i in range(len(moved_polygons)):
plt.plot(np.array(moved_polygons[i].exterior.xy[0]),np.array(moved_polygons[i].exterior.xy[1]))#'k-')
plt.xlabel(r"x [$\mu$m]")
plt.ylabel(r"y [$\mu$m]")
plt.axis("equal")
plt.show()
[22]:
testmap=stack.map_from_polygons(moved_polygons,blending="average")
plt.imshow(testmap)
plt.colorbar()
plt.show()
100%|██████████| 25/25 [00:00<00:00, 516.79it/s]
Optional: second iteration (end)
[23]:
mda.imsave("stitching_example_small.png",testmap.astype(np.uint8))
(2096, 2206)
[23]:
True