Remove only black spots in scanned documents?

Caleb13 · **Joined:** Jun 26, 2024 **Posts:** 8

Edit: Trandoductin wrote a GIMP plugin that does exactly what I needed. The best-working version is currently in this post.

I also wrote a script for image-processing package ImageJ, you can get it in this post. It works much faster than GIMP, it took only about 2 seconds per image in my case.

Hello, I occassionaly use GIMP with BIMP plugin to mass-process scanned documents. Now I have hundreds of monochrome (1-bit) pages that have lots of isolated black spots. Unfortunately, they can't be avoided, because the source documents were printed on cheap paper with visible wood pieces. I know there are complicated manual methods how I could remove them, but they can't be effectively automated with BIMP.

So I tried to use G'MIC to remove them with "Despeckle", "Remove Hot Pixels" and "Repair Scanned Document". However, all these filters have a common problem: they also remove isolated white spots. I'm attaching a screenshot which showcases it - the filters fill the insides of number "4", which is highly undesirable. Is there some trick to remove only black spots?

Also, G'MIC doesn't work on 1-bit images, I have to convert them to grayscale first. I don't like doing that, because some filters may blur the edges, which worsens readability after converting back to 1-bit. Is there some way around it?

Thanks!

ofnuts · **Joined:** Oct 25, 2010 **Posts:** 4812

* Color-select Select the white
* Select > Grow by one pixel (or two, if you dare)
* Select > Shrink by one pixel (or two, id you dared)
* Bucket-fill selection with white.

Caleb13 · **Joined:** Jun 26, 2024 **Posts:** 8

I tried the grow/shrink method, but it's unusable, too many parts of text disappear.

I googled a way to do what I want, but it requires image processing program ImageJ that's usually used in medicine, tissue analysis and related fields. The results are very good, but it requires about 10 steps to get there, most importantly Analyze Particles function. I found it in thread "Is there a way to delete lonely pixels?" on Reddit. (This forum doesn't allow me to post links, because I'm a new user.) And unfortunately, making ImageJ batch-process multiple files is rather complicated.

Anyway, I'm attaching an image that compares the results.

PixLab · **Posted:** Mon Jul 01, 2024 7:44 am **(#4)**

G'MIC > Details > Spotify to new layer

Attachment:

screenshot_20240701-203421.jpg [ 187.45 KiB | Viewed 10205 times ]

On the new black layer select with the fuzzy select tool in mode addition the big black part (no more specs...)

Attachment:

screenshot_20240701-203715.jpg [ 219.72 KiB | Viewed 10205 times ]

Then select the white layer in the layers' dialog and just drag and drop the white color from the FG/BG icon then Select > None

Result in one minute

Attachment:

screenshot_20240701-204320.jpg [ 188.65 KiB | Viewed 10205 times ]

Steve · **Joined:** Dec 26, 2014 **Posts:** 205

You might be able to make use of the script in the zipped folder below which converts single black pixels to white. It was written for use with RGB (no alpha channel) image mode so you would have to convert your mono image to RGB, run the script then convert back.

Please note for the script to work, the converted black pixels must be black 0,0,0 and the 8 pixels surrounding it must all be white 255,255,255

If you wish to try it place the script in your plug-ins folder, restart Gimp and it can be found at menu bar, '<Image>/Filters/Remove single black pixels'

It works on my Gimp 2.10.36

Hope it works for you

EDIT I've just noticed an omission from the script :oops:

Before running the script ensure the background colour is set to white 255, 255, 255. If it isn't any single black pixels on the very edge of the image will remain

Sorry for this

Attachment:

remove-single-black-pixels.zip [924 Bytes]
Downloaded 113 times

Caleb13 · **Joined:** Jun 26, 2024 **Posts:** 8

That Spotify trick is clever, but can't be automated because of the Fuzzy tool. (Hmm, "Spotify"... that became kind of confusing name today, hasn't it?) So I bit the bullet and made a macro for ImageJ, based on that Reddit thread. Actually, I made two versions of the macro: one standalone and second for its built-in batch tool.

The built-in batch tool is more user-friendly, you can open it via Process -> Batch -> Macro from the main menu. Then you must select input and output directories, output file format and paste this macro code (see screenshot below):

Code: Select all

filename = getTitle();     
run("8-bit");
setAutoThreshold("Default");
setOption("BlackBackground", true);
run("Convert to Mask");
run("Analyze Particles...", "size=0-10 show=Masks clear");
MaskName = "Mask of " + filename;
imageCalculator("XOR create", MaskName, filename);

Input and output directories must be different, because output files have the same name as input ones. If you click Process button, seemingly nothing will happen. But you may see what ImageJ does as texts under the main menu icons. You can change size of removed spots with "size=0-10" parameter. Don't overdo it, otherwise ImageJ will also remove small punctuation marks or diacritics. Remember that ImageJ can't save monochrome (1-bit) images, it produces 256-color indexed PNGs in my case. I had to convert them back to monochrome with GIMP+BIMP afterwards.

I'm also attaching full macro "Departicle.ijm", you can run it via Plugins -> Macro -> Run. But it's even more opaque, it doesn't report any errors it encounters, for example. And you will have to manually edit paths to input and output directories. However, it adds "departicled" suffix to the output files, so you can use the same directory.

Code: Select all

input = "D:/Scan in/";
output = "D:/Scan out/";

function action(input, output, filename) {
    open(input + filename);
    run("8-bit");
   setAutoThreshold("Default");
   setOption("BlackBackground", true);
   run("Convert to Mask");
   run("Analyze Particles...", "size=0-10 show=Masks clear");
   MaskName = "Mask of " + filename;
   imageCalculator("XOR create", MaskName, filename);
   saveAs("PNG", output + filename + " departicled");
   close("*");
}

setBatchMode(true); 
list = getFileList(input);
for (i = 0; i < list.length; i++){
        action(input, output, list[i]);
}
setBatchMode(false);

karo03de · **Joined:** Jul 02, 2013 **Posts:** 59

I don't know how far your G'MIC knowledge reaches.

I tried in command mode your image with

$ gmic Downloads/blackspots.png split c keep\[0\] negate +karo_segmentarea 6 negate

where areas less 6 pixels are deleted...

PixLab · **Posted:** Tue Jul 09, 2024 6:37 am **(#8)**

karo03de wrote:

$ gmic Downloads/blackspots.png split c keep\[0\] negate +karo_segmentarea 6 negate

Very nice, but how do you save it?

I'm asking because no right click, no "conventional" keyboard shortcut, like Ctrl+s/x/c

Attachment:

screenshot_20240709-193259.jpg [ 117.56 KiB | Viewed 9975 times ]

karo03de · **Joined:** Jul 02, 2013 **Posts:** 59

e.g.

gmic Downloads/blackspots.png split c keep\[0\] negate +karo_segmentarea 6 negate to_rgb. o. Download/noblackspots.png

or overwriting the input file

gmic Downloads/blackspots.png split c keep\[0\] negate +karo_segmentarea 6 negate to_rgb. ow.

Do gmic help o or gmic help ow!

By the way ^S or ^O should save, look for gmic help display

PixLab · **Posted:** Tue Jul 09, 2024 8:50 am **(#10)**

karo03de wrote:

e.g.
gmic Downloads/blackspots.png split c keep\[0\] negate +karo_segmentarea 6 negate to_rgb. o. Download/noblackspots.png
or overwriting the input file
gmic Downloads/blackspots.png split c keep\[0\] negate +karo_segmentarea 6 negate to_rgb. ow.
Do gmic help o or gmic help ow!
By the way ^S or ^O should save, look for gmic help display

Thanks, but "to_rgb. o. Download/noblackspots.png" is not working the result is a black image

I see a huge potential to something like this

Code: Select all

for i in *; 
do gmic $i split c keep\[0\] negate +karo_segmentarea 6 negate to_rgb. o. Done/$i; 
done

karo03de · **Joined:** Jul 02, 2013 **Posts:** 59

The result is 0,1 scaled. Multiply by 255

mul. 255 to_rgb. o.

By the way, be careful, input should be binary. Your for loop will have some interdependencies of shell and gmic script! Try better gmic run "....".

PixLab · **Posted:** Tue Jul 09, 2024 8:20 pm **(#12)**

karo03de wrote:

The result is 0,1 scaled. Multiply by 255
mul. 255 to_rgb. o.
By the way, be careful, input should be binary. Your for loop will have some interdependencies of shell and gmic script! Try better gmic run "....".

Hum, still not working as expected, or today terminal woke up on the wrong side of the bed :mrgreen:

I did

Code: Select all

gmic a.png split c keep\[0\] negate +karo_segmentarea 6 negate mul. 255 to_rgb. o. w.png

Before:

Attachment:

a.png [ 54.14 KiB | Viewed 9922 times ]

after:

Attachment:

w.png [ 10.56 KiB | Viewed 9922 times ]

karo03de · **Joined:** Jul 02, 2013 **Posts:** 59

Your example image is not binary. You could try

gmic a.png s c k[0] gt 200 negate +karo_segmentarea 6 negate mul. 255 to_rgb. o. w.png

Depending on your input possibly other means to prepare a binary image are necessary.

trandoductin · **Posted:** Wed Jul 10, 2024 8:28 am **(#14)**

Not sure if you found solution or not but I got this working with default setting of 12 threshold and 100 maxpixels

Attached is the zipped version of code (below):

Code: Select all

#!/usr/bin/env python
# Author: (Tin Tran)
# Created On: 2024-07-10
# Remove only black spots in scanned documents?
# Possible Solution: use logics from http://gimpchat.com/viewtopic.php?f=9&t=14045&p=256687&hilit=DivideTransBG.scm#p256687
# which can divide up transparent background to get alpha shapes. Except this problem is just white
# so we'll let user select background color, threshold, and replace all shapes that have less than minimum allowed pixels
#
# License: Whatever GIMP's License is.
# Revisions:
# Rel 0.10: Initial Version
from gimpfu import *
import random
import math


def remove_dots(image,layer,bgcolor,threshold,maxpixels):

    #interpolation none scale image so that dots are preserved when we
    #selection to path it
    pdb.gimp_image_scale_full(image,image.width*4,image.height*4,0)
    pdb.gimp_by_color_select(layer,bgcolor,threshold,CHANNEL_OP_REPLACE,TRUE,FALSE,0,FALSE)
    pdb.plug_in_sel2path(image,layer)
    active_vectors = pdb.gimp_image_get_active_vectors(image)
    num_strokes,stroke_ids = pdb.gimp_vectors_get_strokes(active_vectors)
    for i in range(0,len(stroke_ids)):
        _type,num_points,controlpoints,closed = pdb.gimp_vectors_stroke_get_points(active_vectors,stroke_ids[i])
        new_vectors = pdb.gimp_vectors_new(image,"single vector")
        pdb.gimp_image_insert_vectors(image,new_vectors,None,0)
        pdb.gimp_vectors_stroke_new_from_points(new_vectors,_type,num_points,controlpoints,closed)
        pdb.gimp_image_select_item(image,CHANNEL_OP_REPLACE,new_vectors)
        mean,std_dev,median,pixels,count_,percentile = pdb.gimp_histogram(layer,HISTOGRAM_VALUE,0,255)
        #if pixel count is less than or equal to our maximum fill it with color
        if count_ <= maxpixels*4:
            pdb.gimp_context_set_foreground(bgcolor)
            pdb.gimp_edit_fill(layer,FILL_FOREGROUND)
        #remove when done with this one
        pdb.gimp_image_remove_vectors(image,new_vectors)
    pdb.gimp_selection_none(image)
    pdb.gimp_image_scale_full(image,image.width/4,image.height/4,0)

register(
    "python_fu_remove_dots",
    "remove dots",
    "remove dots by filling in with bg color",
    "author name",
    "copyright name",
    "2024.07.10",
    "Remove Dots...",
    "RGB*",      # Alternately use RGB, RGB*, GRAY*, INDEXED etc.
    [
    #INPUT BEGINS
    (PF_IMAGE, "image", "Image", None),
    (PF_DRAWABLE,   "layer", "Drawable", None),
    (PF_COLOR, "bgcolor", "Background Color:", (255, 255, 255) ), # extra param is RGB triple
    (PF_SLIDER, "threshold", "Threshold:", 12, (0, 100, 1)),
    (PF_SLIDER, "maxpixels", "Maximum No Of Pixels To Fill With BGColor:", 100, (0, 500, 1)),
    #INPUT ENDS
    ],
    [],
    remove_dots,
    menu="<Image>/Python-Fu")

main()

# Below is all the example input types for INPUTS for the plug-in which can be cut and pasted into #INPUT BEGINS section and edited to taste (found online years back that I didn't want to constantly look up)
# Since GIMP is free anyways, I thought it would be handy here for me to CUT and PASTE, CHANGE, USE.
#           (PF_INT, "p0", "_INT:", 0), # PF_INT8, PF_INT16, PF_INT32  similar but no difference in Python.
#           (PF_FLOAT, "p02", "_FLOAT:", 3.141),
#           (PF_STRING, "p03", "_STRING:", "foo"),  # alias PF_VALUE
#           (PF_TEXT, "p04", "TEXT:", "bar"),
#           # PF_VALUE
#           # Pick one from set of choices
#           (PF_OPTION,"p1",   "OPTION:", 0, ["0th","1st","2nd"]), # initially 0th is choice
#           (PF_RADIO, "p16", "RADIO:", 0, (("0th", 1),("1st",0))), # note bool indicates initial setting of buttons
#           # PF_RADIO is usually called a radio button group.
#           # SLIDER, ADJUSTMENT types require the extra parameter of the form (min, max, step).
#           (PF_TOGGLE, "p2",   "TOGGLE:", 1), # initially True, checked.  Alias PF_BOOL
#           # PF_TOGGLE is usually called a checkbox.
#           (PF_SLIDER, "p3", "SLIDER:", 0, (0, 100, 10)),
#           (PF_SPINNER, "p4", "SPINNER:", 21, (1, 1000, 50)),  # alias PF_ADJUSTMENT
#           # Pickers ie combo boxes ie choosers from lists of existing Gimp objects
#           (PF_COLOR, "p14", "_COLOR:", (100, 21, 40) ), # extra param is RGB triple
#           # PF_COLOUR is an alias by aussie PyGimp author lol
#           (PF_IMAGE, "p15", "IMAGE:", None), # should be type gimp.image, but None works
#           (PF_FONT, "p17", "FONT:", 0),
#           (PF_FILE, "p18", "FILE:", 0),
#           (PF_BRUSH, "p19", "BRUSH:", 0),
#           (PF_PATTERN, "p20", "PATTERN:", 0),
#           (PF_GRADIENT, "p21", "GRADIENT:", 0),
#           (PF_PALETTE, "p22", "PALETTE:", 0),
#           (PF_LAYER, "p23", "LAYER:", None),
#           (PF_CHANNEL, "p24", "CHANNEL:", None),  # ??? Usually empty, I don't know why.
#           (PF_DRAWABLE, "p25", "DRAWABLE:", None),
#           # Mostly undocumented, but work
#           (PF_VECTORS, "p26", "VECTORS:", None),
#           (PF_FILENAME, "p27", "FILENAME:", 0),
#           (PF_DIRNAME, "p28", "DIRNAME:", 0)
#           # PF_REGION might work but probably of little use.  See gimpfu.py.

Better version

trandoductin · **Posted:** Wed Jul 10, 2024 8:37 am **(#15)**

minor math error (I scale the image to be 4 times width and 4 times the height before doing work on it) so it should be applied as maxpixels *4 *4.
Here's the fix for that and default should now be 12(threshold) and 25 for maxpixels.
I check to see if it removes the comma and it doesn't so that's a good number.

Logics if you're interested:
1. Scale image to be 4 timesx width and height (no interpolation).
2. Select by background color.
3. Turn it into path.
4. Select each stroke and create a path out of that and use histograms to count pixels of selection of path.
5. if it's less then equal to max pixels, fill it with bgcolor.
6. when done, scale back to original size.

Code: Select all

#!/usr/bin/env python
# Author: (Tin Tran)
# Created On: 2024-07-10
# Remove only black spots in scanned documents?
# Possible Solution: use logics from http://gimpchat.com/viewtopic.php?f=9&t=14045&p=256687&hilit=DivideTransBG.scm#p256687
# which can divide up transparent background to get alpha shapes. Except this problem is just white
# so we'll let user select background color, threshold, and replace all shapes that have less than minimum allowed pixels
#
# License: Whatever GIMP's License is.
# Revisions:
# Rel 0.10: Initial Version
from gimpfu import *
import random
import math


def remove_dots(image,layer,bgcolor,threshold,maxpixels):

    #interpolation none scale image so that dots are preserved when we
    #selection to path it
    pdb.gimp_image_scale_full(image,image.width*4,image.height*4,0)
    pdb.gimp_by_color_select(layer,bgcolor,threshold,CHANNEL_OP_REPLACE,TRUE,FALSE,0,FALSE)
    pdb.plug_in_sel2path(image,layer)
    active_vectors = pdb.gimp_image_get_active_vectors(image)
    num_strokes,stroke_ids = pdb.gimp_vectors_get_strokes(active_vectors)
    for i in range(0,len(stroke_ids)):
        _type,num_points,controlpoints,closed = pdb.gimp_vectors_stroke_get_points(active_vectors,stroke_ids[i])
        new_vectors = pdb.gimp_vectors_new(image,"single vector")
        pdb.gimp_image_insert_vectors(image,new_vectors,None,0)
        pdb.gimp_vectors_stroke_new_from_points(new_vectors,_type,num_points,controlpoints,closed)
        pdb.gimp_image_select_item(image,CHANNEL_OP_REPLACE,new_vectors)
        mean,std_dev,median,pixels,count_,percentile = pdb.gimp_histogram(layer,HISTOGRAM_VALUE,0,255)
        #if pixel count is less than or equal to our maximum fill it with color
        if count_ <= maxpixels*4*4:
            pdb.gimp_context_set_foreground(bgcolor)
            pdb.gimp_edit_fill(layer,FILL_FOREGROUND)
        #remove when done with this one
        pdb.gimp_image_remove_vectors(image,new_vectors)
    pdb.gimp_selection_none(image)
    pdb.gimp_image_scale_full(image,image.width/4,image.height/4,0)

register(
    "python_fu_remove_dots",
    "remove dots",
    "remove dots by filling in with bg color",
    "author name",
    "copyright name",
    "2024.07.10",
    "Remove Dots...",
    "RGB*",      # Alternately use RGB, RGB*, GRAY*, INDEXED etc.
    [
    #INPUT BEGINS
    (PF_IMAGE, "image", "Image", None),
    (PF_DRAWABLE,   "layer", "Drawable", None),
    (PF_COLOR, "bgcolor", "Background Color:", (255, 255, 255) ), # extra param is RGB triple
    (PF_SLIDER, "threshold", "Threshold:", 12, (0, 100, 1)),
    (PF_SLIDER, "maxpixels", "Maximum No Of Pixels To Fill With BGColor:", 25, (0, 500, 1)),
    #INPUT ENDS
    ],
    [],
    remove_dots,
    menu="<Image>/Python-Fu")

main()

# Below is all the example input types for INPUTS for the plug-in which can be cut and pasted into #INPUT BEGINS section and edited to taste (found online years back that I didn't want to constantly look up)
# Since GIMP is free anyways, I thought it would be handy here for me to CUT and PASTE, CHANGE, USE.
#           (PF_INT, "p0", "_INT:", 0), # PF_INT8, PF_INT16, PF_INT32  similar but no difference in Python.
#           (PF_FLOAT, "p02", "_FLOAT:", 3.141),
#           (PF_STRING, "p03", "_STRING:", "foo"),  # alias PF_VALUE
#           (PF_TEXT, "p04", "TEXT:", "bar"),
#           # PF_VALUE
#           # Pick one from set of choices
#           (PF_OPTION,"p1",   "OPTION:", 0, ["0th","1st","2nd"]), # initially 0th is choice
#           (PF_RADIO, "p16", "RADIO:", 0, (("0th", 1),("1st",0))), # note bool indicates initial setting of buttons
#           # PF_RADIO is usually called a radio button group.
#           # SLIDER, ADJUSTMENT types require the extra parameter of the form (min, max, step).
#           (PF_TOGGLE, "p2",   "TOGGLE:", 1), # initially True, checked.  Alias PF_BOOL
#           # PF_TOGGLE is usually called a checkbox.
#           (PF_SLIDER, "p3", "SLIDER:", 0, (0, 100, 10)),
#           (PF_SPINNER, "p4", "SPINNER:", 21, (1, 1000, 50)),  # alias PF_ADJUSTMENT
#           # Pickers ie combo boxes ie choosers from lists of existing Gimp objects
#           (PF_COLOR, "p14", "_COLOR:", (100, 21, 40) ), # extra param is RGB triple
#           # PF_COLOUR is an alias by aussie PyGimp author lol
#           (PF_IMAGE, "p15", "IMAGE:", None), # should be type gimp.image, but None works
#           (PF_FONT, "p17", "FONT:", 0),
#           (PF_FILE, "p18", "FILE:", 0),
#           (PF_BRUSH, "p19", "BRUSH:", 0),
#           (PF_PATTERN, "p20", "PATTERN:", 0),
#           (PF_GRADIENT, "p21", "GRADIENT:", 0),
#           (PF_PALETTE, "p22", "PALETTE:", 0),
#           (PF_LAYER, "p23", "LAYER:", None),
#           (PF_CHANNEL, "p24", "CHANNEL:", None),  # ??? Usually empty, I don't know why.
#           (PF_DRAWABLE, "p25", "DRAWABLE:", None),
#           # Mostly undocumented, but work
#           (PF_VECTORS, "p26", "VECTORS:", None),
#           (PF_FILENAME, "p27", "FILENAME:", 0),
#           (PF_DIRNAME, "p28", "DIRNAME:", 0)
#           # PF_REGION might work but probably of little use.  See gimpfu.py.

Demo video:

Video from : www.youtube.com

Better version in post #35

karo03de · **Joined:** Jul 02, 2013 **Posts:** 59

Interesting python solution.

I still stick with G'MIC, there is a s.t.s. standard function area_fg replacing my community solution karo_segmentarea generating from the binary image one setting the pixel value of each connected object to its area:

gmic Downloads/a.png s c k[0] gt 150 negate +area_fg 0 ge. 6 ...

trandoductin · **Posted:** Wed Jul 10, 2024 10:56 am **(#17)**

Thanks, and Thank you OP for posting the problem for me to have opportunity to scratch my "I am itching to write a plug-in" itch.

sallyanne · **Posted:** Wed Jul 10, 2024 10:55 pm **(#18)**

lol tin

trandoductin · **Posted:** Thu Jul 11, 2024 8:33 am **(#19)**

trandoductin · **Posted:** Thu Jul 11, 2024 8:33 am **(#20)**

what sucks is I am itching more now...