I first started the assignment by thinking about placing a template all over the map, but I didn't like the idea of taking pixel measurements and then trying out the template everywhere. I felt like it would take the program too long and felt very hard-coded. So I instead used the intensity filter that Professor Atkeson provided and heightened the threshold to remove a little more. Then I added my own filter to remove any straggling pixels that were alone and away from the block, leaving only the block. I provided a video for you to see and understand what I talk about next. My program then detects any edges by checking if a pixel has any neighbors that are 0 (due to the intensity filter) and marks them green on the image and then stores them in a matrix. After creating that matrix I find the min and max values within it, which are the corners of the rectangle that inscribes the block in the image and then I draw it and make matrices out of those lines in the image. If any of the edges in the edge matrix are within the lines I make them the corners (in the code they're called tangentTop, tangentBot, etc. since they're the points that touch the line) and plot them as stars. Next I find the centroid by finding the midpoint between the corners of the inscribing rectangle and plot it with a red star.
Next is the orientation of the block, which I did use a template of sorts. In the video, after the rectangle and corners are drawn, a couple of green lines are drawn. If the green lines match up with the block well then the block is either in 0 rads or pi/2 rads. I check which one by seeing if the bottom line is longer or not, if it is then the block is at 0 rads, else the bottom line is shorter and the block is standing up and at pi/2 rads. The line that's longer and shows that the block is at 0 or pi/2 rads then lights up to yellow. I used a template for this because due to the flatness of the block in this position, there are too many tangential points to the line I talked about earlier, which confuses the corners.
After the template I check the angle between two points, the top and left or bottom and left, depending on which side is the longer side of the block. In the video you can see that a red line is made between the two points. Then I use atan2 with the differences in y and x between the points to find the angle of orientation of the block, which then finishes the program.
I went through that process rather sequentially in that I didn't have to backtrack and change the design (for the most part), besides the beginning when I tried using a template. Though there was a big issue in trying to get the corners at first. I tried finding a group of edge pixels that were two intersecting lines, but the code for it seemed to complicated, and I felt like there was a better way. I then realized that I created a rectangle with the outer corners that inscribe the block and used to the edges to find tangential edge points that became the corners of the block.
The accuracy of the centroid is probably the most accurate measurement in my program since I first find the rectangle that the block in the image is inscribed in, then take the centroid of that. So probably to within a couple millimeters. The corners of the block that I measure out tend to be a bit off due to how much I filter the image. Some of the images have some of a corner or side filtered out so the corner my program computes isn't the true corner, but one that is very nearby(usually), but in some cases a bit far away, probably to within a centimeter or two. The angle is taken along the long side of the board which I think is accurate to a rather small error, perhaps to within .1-.01 rads.
I didn't have to use the depth array for centroid and theta part of the first assignment, just the rgba images. However, missing depth values could be treated in the same way that filtered out rgba values (which are also basically 0) in that we ignore them considering how many data points we still have.
Slight change in code: centroid was returning as (y,x) instead of (x,y), so I fixed that.