Some notes on OpenCV findFundamentalMat

27 Sep 2012 | Programming

These are some notes about OpenCV’s findFundamentalMat function. It all started while trying to write unittests that rely on the fundamental matrix. I created a set of points matches projected from an artificial shape (so they are near perfect matches) and realized that depending on the number of points, I was getting very different results from findFundamentalMat (in term of error).

I wrote a quick script to test findFundamentalMat function. I also wanted to test if it handled 32 and 64 bit floats/double correctly (because initially, I thought the bad results were due to some float/double conversion bug between python and C++).

import cv2
import numpy as np
import numpy.linalg as la

# 15 Points
p1 = [[245.77, 248.66, 263.39, 251.11, 249.88, 287.57, 240.93, 206.84,
       224.95, 231.47, 257.55, 264.71, 227.67, 246.01, 244.15],
      [169.57, 105.82, 182.45, 146.54, 197.99, 137.11, 113.15, 171.57,
       170.34, 170.21, 124.43, 176.16, 127.05, 113.26, 154.41]]

p2 = [[267.07, 252.92, 254.22, 284.33, 236.82, 220.04, 255.9, 259.09,
       241.73, 258.62, 277.83, 219.62, 262.91, 250.93, 287.09],
      [172.34, 105.02, 190.25, 145.1, 190.63, 135.11, 114.37, 165.9,
       164.27, 168.3, 112.98, 170.46, 129.03, 114.5, 155.89]]

x1_32 = np.array(p1, dtype='f4')
x1_64 = np.array(p1, dtype='f8')
x2_32 = np.array(p2, dtype='f4')
x2_64 = np.array(p2, dtype='f8')

def compute_F_error(F):
    errs = []
    for i, p in enumerate(zip(x1_32.T, x2_32.T)):
        e = np.r_[p[1], 1].T.dot(F_32).dot(np.r_[p[0], 1])
        # normalize error by the norm of F since F is defined up to scale
        e /= la.norm(F)
        errs.append(e)
    return np.mean(errs)

np.set_printoptions(suppress=True, precision=6)

print '-- 15 points (RANSAC)'
F_32, status = cv2.findFundamentalMat(x1_32.T, x2_32.T)
print "F 32 : ", F_32/F_32[2,2]
print "Avg error : ", compute_F_error(F_32/F_32[2,2])
print "status : ", np.ravel(status)
F_64, status = cv2.findFundamentalMat(x1_64.T, x2_64.T)
print "F 64 : ", F_64/F_64[2,2]
print "Avg error : ", compute_F_error(F_64/F_64[2,2])
print "status : ", np.ravel(status)

print '-- 12 points (LMEDS)'
x1_32 = x1_32[:,:12]
x2_32 = x2_32[:,:12]
x1_64 = x1_64[:,:12]
x2_64 = x2_64[:,:12]
F_32, status = cv2.findFundamentalMat(x1_32.T, x2_32.T)
print "F 32 : ", F_32/F_32[2,2]
print "Avg error : ", compute_F_error(F_32/F_32[2,2])
print "status : ", np.ravel(status)
F_64, status = cv2.findFundamentalMat(x1_64.T, x2_64.T)
print "F 64 : ", F_64/F_64[2,2]
print "Avg error : ", compute_F_error(F_64/F_64[2,2])
print "status : ", np.ravel(status)

print '-- 7 points'
x1_32 = x1_32[:,:7]
x2_32 = x2_32[:,:7]
x1_64 = x1_64[:,:7]
x2_64 = x2_64[:,:7]
F_32, status = cv2.findFundamentalMat(x1_32.T, x2_32.T)
print "F 32 : ", F_32/F_32[2,2]
print "Avg error : ", compute_F_error(F_32/F_32[2,2])
print "status : ", np.ravel(status)
F_64, status = cv2.findFundamentalMat(x1_64.T, x2_64.T)
print "F 64 : ", F_64/F_64[2,2]
print "Avg error : ", compute_F_error(F_64/F_64[2,2])
print "status : ", np.ravel(status)

This program gives the following output. Keep in mind that the fundamental matrix is defined only up to scale. So even if the matrices look very different, they are, in fact, all valid fundamental matrices.

~|=> python cvf.py
-- 15 points (RANSAC)
F 32 :  [[-0.        0.000013 -0.002001]
 [ 0.000013  0.       -0.004486]
 [-0.001997 -0.002184  1.      ]]
Avg error :  3.52578346368e-06
status :  [1 1 1 1 1 1 1 1 1 1 1 1 1 1 1]
F 64 :  [[-0.        0.000013 -0.002001]
 [ 0.000013  0.       -0.004486]
 [-0.001997 -0.002184  1.      ]]
Avg error :  3.52578346367e-06
status :  [1 1 1 1 1 1 1 1 1 1 1 1 1 1 1]
-- 12 points (LMEDS)
F 32 :  [[-0.00002  -0.000007  0.007063]
 [ 0.000093 -0.000031 -0.022699]
 [-0.00931   0.011469  1.      ]]
Avg error :  -0.00957514987137
status :  [1 0 1 0 1 0 0 1 1 1 1 0]
F 64 :  [[-0.        0.000013 -0.001986]
 [ 0.000013  0.       -0.004512]
 [-0.002002 -0.002178  1.      ]]
Avg error :  -0.00957874125736
status :  [1 1 0 1 0 0 1 0 0 1 1 1]
-- 7 points
F 32 :  [[ 0.000023 -0.000003 -0.005589]
 [-0.000007 -0.000006  0.002998]
 [-0.0045    0.001153  1.      ]]
Avg error :  -7.55324825546e-15
status :  [1 1 1 1 1 1 1]
F 64 :  [[ 0.000023 -0.000003 -0.005589]
 [-0.000007 -0.000006  0.002998]
 [-0.0045    0.001153  1.      ]]
Avg error :  -7.55324825538e-15
status :  [1 1 1 1 1 1 1]

As you can see, the average error is ok for the 15 and 7 points case, but is relatively high (with some outliers) for the 12 points case.

If you look at findFundamentalMat’s documentation, it says it uses RANSAC if the number of points is greater or equal to 7. But this is kind of misleading (OpenCV bug) because actually, OpenCV does the following :

So our test with 12 points uses LMeDS, which, according to a paper[1], is not very good when the number of outliers is small (figure 3 of the paper, you can see RANSAC is better until the outliers percentage growth over 50%).

Since this script tests with near-perfect matches, we don’t benefit at all from LMeDS’s robustness to high outliers percentage.

Conclusion : float/double conversion from python to C++ works fine, but be choice of the fundamental matrix estimation algorithm matters.

[1]: Robust Estimation of the Fundamental Matrix and Stereo Correspondences, by Nuno Gracias , José Santos-Victor

comments