This project will generate a video which contains a sequence of the same image from the most blurry to the most focused and back to the most blurry. Use this video as a test sample for the autofocus function to identify the most focused image frame in the video. The AutoFocus methods include Variance of absolute values of Laplacian and Sum Modified Laplacian.
Use the VideoWriter to generate a video with the same focused image.
fourcc = cv2.VideoWriter_fourcc(*'mp4v')
out = cv2.VideoWriter(output_path, fourcc, fps, (inputframe_width, inputframe_height))
for i in range(frame_qty):
out.write(frameImg)
out.release()
cv2.destroyAllWindows()
Reference:
https://docs.opencv.org/3.4/dd/d43/tutorial_py_video_display.html
A focused image will go through a gradual blurring process to generate a sequence of images with the incremented Gaussian blurs.
The GaussianBlur algorithm convolves the source image with the specified Gaussian kernel to smooth the high contrast feature edges. Specify the width and height of the kernel which should be positive and odd. The Gaussian kernel standard diviation in both X and Y directions can also be specified.
Reference:
https://docs.opencv.org/4.x/d4/d86/group__imgproc__filter.html
fourcc = cv2.VideoWriter_fourcc(*'mp4v')
out = cv2.VideoWriter(output_path, fourcc, fps, (frame_width, frame_height))
kernel_size = 1
while True:
ret, frame = cap.read()
if not ret:
break
if kernel_size <= max_kernel_size:
blurred_frame = cv2.GaussianBlur(frame, (kernel_size, kernel_size), 0)
out.write(blurred_frame)
kernel_size += blur_increment
else:
out.write(cv2.GaussianBlur(frame, (max_kernel_size, max_kernel_size), 0))
cap.release()
out.release()
cv2.destroyAllWindows()
frames = []
while True:
ret, frame = cap.read()
if not ret:
break
frames.append(frame)
cap.release()
Simply apply the [reversed] command to the list of frames.
fourcc = cv2.VideoWriter_fourcc(*'mp4v')
if frames:
height, width, _ = frames[0].shape
out = cv2.VideoWriter(output_path, fourcc, fps, (width, height))
for frame in reversed(frames):
out.write(frame)
out.release()
else:
print("The video has no frames")
Collect the three videos' file paths to a list. Read each target video name from the list and write to a single output video to concatenate the three videos.
video_captures = [cv2.VideoCapture(path) for path in video_paths]
fourcc = cv2.VideoWriter_fourcc(*'mp4v') # Codec
output_writer = cv2.VideoWriter(output_path, fourcc, fps, (frame_width, frame_height))
# Iterate through each video and append its frames
for capture in video_captures:
while(capture.isOpened()):
ret, frame = capture.read()
if ret:
output_writer.write(frame)
else:
break
capture.release()
output_writer.release()
cv2.destroyAllWindows()
The goal of this project is to use the image filtering techniques to find the sharpest frame in the target video which goes from blurry to focuse to blurry.
The Laplacian operator's elements were mapped with the pixel locations. The coefficients of the Lapacian operator can be used as kernel cell values corresponding to the pixel locations.
It equivalently measures the rate at which pixel intensity changes in an image such as edge detection.
The Laplacian operator is
\begin{equation}
\frac{\partial^2 f}{\partial^2 x} + \frac{\partial^2 f}{\partial^2 y}
=f(x+1, y)+f(x-1, y)+f(x, y+1)+f(x, y-1)-4 \times f(x,y)
\end{equation}
The image convolution kernel becomes:
\begin{bmatrix} 0 & 1 & 0 \\ 1 & -4 & 1 \\ 0 & 1 & 0 \end{bmatrix}
Use the variance of new values after convolution as a new focus measure. The variance value is used as a threshold to determine whether the image is blurry or sharp. The sharp image has a high variance value.
def LaplacianVariance(image):
#Convert the color image to grayscale
imgray=cv2.cvtColor(image,cv2.COLOR_BGR2GRAY)
#Calculate the laplacian matrix of the grayscale image
lap = cv2.Laplacian(imgray, cv2.CV_32F, ksize = 3,
scale = 1, delta = 0)
#Calculate the variance at each pixel location
delta = lambda t: (np.abs(t)-np.mean(np.abs(lap)))**2
var_lap_array = np.array([delta(xi) for xi in lap])
#Sum all variances
return np.sum(var_lap_array)
In the Laplacian operator, the second derivatives in the x and y directions can have opposite signs and result in cancellation of their values.
The Sum Modified Laplacian method uses the sum of the absolue values of the second derivatives.
The step variable specifies the spacing between target pixels.
The focus measure is the sum of these modified Lapacian values, which are greater than a threshold value.
Reference:
https://cave.cs.columbia.edu/Statics/publications/pdfs/Nayar_TR89.pdf
def Modified_Laplacian(im):
step=3
thresh=7
gy=cv2.cvtColor(im,cv2.COLOR_BGR2GRAY)
(wx,wy)=gy.shape
ML=[]
for i in range(step,wx-step):
for j in range(step,wy-step):
sml=abs(2*gy[i,j]-gy[i-step,j]-gy[i+step,j])+ \
abs(2*gy[i,j]-gy[i,j-step]-gy[i,j+step])
if sml>=thresh:
ML.append(sml)
return sum(ML)
Frame ID of the best frame [Variance of Absolute Laplacian]: 109
Frame ID of the best frame [Sum of Modified Laplacian]: 110
The identified best focused image frames are shown below:
