It is the 3×3 Sobel edge filter.

The source is in the filter() method of the ij/process/ByteProcessor.java class.

...
case FIND_EDGES: // 3x3 Sobel filter
sum1 = p1 + 2*p2 + p3 - p7 - 2*p8 - p9;
sum2 = p1  + 2*p4 + p7 - p3 - 2*p6 - p9;
sum = (int)Math.sqrt(sum1*sum1 + sum2*sum2); 
if (sum> 255) sum = 255;
break;
...

p1…p9 hold the grey value of the pixels in the 3×3 kernel (the central pixel is p5).

There are many other edge detecting algorithms (such as Canny, Roberts, Marr's Laplacian-of-Gaussian or LoG, difference-of-boxes or DoB, Frei-Chen, etc.).