How to find duplicate photos in a folder

Compute a hash of every photo, and any two files with the same hash are exact copies. Move the extra copies into a review folder first, look through it, and only delete it once you are sure.

A hash is a short fingerprint of a file's contents. Two files with the same SHA-256 hash are the same file, whatever their names or dates. This finds exact copies only: a resized, edited or re-saved photo has a different hash, even if it looks the same.

Before you start

1. Back up the folder. Duplicate cleanup is deleting, so have a copy you can return to.

1. Go to the folder.

cd ~/Pictures

Mac and Linux

1. List the duplicate copies. This finds JPEG, PNG and HEIC files in this folder and below, hashes them, and prints every file whose contents already appeared earlier in the list.

On Linux:

find . -type f \( -iname '*.jpg' -o -iname '*.jpeg' -o -iname '*.png' -o -iname '*.heic' \) \
  -exec sha256sum {} + | sort |
  awk '{h=substr($0,1,64)} seen[h]++ {print substr($0,67)}' > duplicates.txt

On a Mac, use shasum -a 256 in place of sha256sum:

find . -type f \( -iname '*.jpg' -o -iname '*.jpeg' -o -iname '*.png' -o -iname '*.heic' \) \
  -exec shasum -a 256 {} + | sort |
  awk '{h=substr($0,1,64)} seen[h]++ {print substr($0,67)}' > duplicates.txt

Both tools print a 64-character hash, two spaces, then the path. The awk part keeps the first file of each hash and lists the others. On a large library this takes a while.

1. Check the list.

wc -l duplicates.txt
head duplicates.txt

Open a few of the listed files next to their originals to be sure.

1. Move the copies to a review folder, outside the folder you are scanning:

mkdir -p ~/duplicates-review
while IFS= read -r f; do
  mv -n -- "$f" ~/duplicates-review/
done < duplicates.txt

-n skips a file if one with the same name is already in the review folder, so nothing is overwritten. Run the loop again later for any skipped files, or rename them first.

1. Review, then delete. Look through ~/duplicates-review. After a few days, if nothing is missing, delete the folder.

Windows (PowerShell)

1. Find the duplicates.

$dupes = Get-ChildItem -Recurse -File -Include *.jpg, *.jpeg, *.png, *.heic |
  Get-FileHash -Algorithm SHA256 |
  Group-Object Hash |
  Where-Object Count -gt 1 |
  ForEach-Object { $_.Group | Select-Object -Skip 1 }
$dupes | Select-Object Path

This keeps the first file in each group and lists the rest.

1. Preview the move, then do it.

New-Item -ItemType Directory -Force "$HOME\duplicates-review" | Out-Null
$dupes | ForEach-Object { Move-Item -LiteralPath $_.Path -Destination "$HOME\duplicates-review" -WhatIf }

-WhatIf shows what would move. Run it again without -WhatIf to move the files. Move-Item stops with an error rather than overwrite a file of the same name.

Which copy to keep

The commands above keep whichever file comes first. If you care which copy stays, for example the one in a dated album folder, sort or filter the list before you move anything, or move by hand from duplicates.txt. For photos that look alike but are not identical, you need a tool that compares images visually, and the results always deserve a human look.

Doing it with Mel

In Mel you can run every command above at the prompt, and each result is a block you can scroll, filter and copy. File paths in the output become links: ctrl+click (cmd+click on macOS) opens the photo, which makes spot checks quick.

You can also ask the agent:

Find exact duplicate photos under ~/Pictures by comparing SHA-256 hashes.
Keep the copy in the oldest folder. Show me how many duplicates you found
and a sample of 10 pairs. Then move the extra copies to
~/duplicates-review. Do not delete anything.

The agent can pick the right hash command for your system, show you the list, and run the move. With Mel's approvals, the move shows on an approval card before it runs, so you decide. Mel's agent runs these commands on your machine. What reaches the model is what the agent reads into the conversation, here a list of file names and hashes, not the photos.

FAQ

How do I find duplicate photos on a Mac? Use shasum -a 256 with find as in step 3, then move the copies to a review folder.

How do I find duplicate files in Windows? In PowerShell, use Get-FileHash and Group-Object Hash, as in step 7.

Will this find edited or resized copies? No. Hashes only match identical files. Similar-looking photos need a visual comparison tool.

Is it safe to delete duplicates automatically? Move them to a review folder first, check it, and delete later. That way a mistake costs nothing.

Why not compare file names or sizes? Different photos can share a name, and copies can have different names. Size is a quick first filter, but only a hash proves two files are the same.

• How to find large files from the command line
• How to rename many files at once
• How to approve or block commands an AI agent runs
• What is an AI desktop agent?