Call to volunteer to build coin search by image

Hello,

Reading the suggestion about Instant Identifier reminds me of an old idea which I was not able to implement yet: search a coin by image
I'm sure this would be a big added value of the catalogue, to help identifying coins.
Similarly to the search by image of Google or TinEye, you would upload a picture of your coin, and the search would show the page of the Numista catalogue corresponding to that coin.

I tried to develop this feature a few years ago, using several methods like SIFT or methods derived from Reisert et al., but I could not reach a "good" accuracy.
I know there has been several other attempts so far, including Collector's Eye and Lookzee, but I could not find a good one for world coins.
Still image recognition is making fast progress, and I can see there are more recent research articles, so I'm sure a coin recognition engine is achievable.

I don't have time to work on that myself, but I strongly believe it would be a unique feature for Numista, and it would be an interesting challenge for anyone studying or working in image processing, machine learning or computer science.
If you are interesting to work on an image-based coin recognition feature for Numista, please reply to this post or send me a message.

I think some sort of text recognition might be a good idea? It's quite different than depending fully on image recognition though.

Great idea nevertheless, bonne chance!

Well, I may be interested in this one. I have some basic experience with OpenCV and image processing.

I don't see how this is doable without a huge multidisciplinary team and budget. (Note that almost all the research focuses on very small numbers of outputs)

First and foremost you'd need an ANN that can scour millions and millions of images. Most of the production quality work has been done for specific coin sets and machines, to learn to identify just 1-20 coins. You'd need it to identify so many not only would you have to train it, you'd have to train it to train different iterations.

Tie the ANN into some of the other methods learned in the latest article you listed and it could work, but you'd have to feed the results of that data back into the ANN too for it to properly work.

It'd be an interesting project on an enterprise level, but it's the kind of thing somebody would make billions if they got it right. Think of the applications, from facial recognition to driverless cars to predicting stocks.

Consider that entire fintech companies dedicate divisions to doing this, and they work with way less hidden layers than you'd need.

Let's say somebody uploads a random round coin.

That's your input layer

You would use some of your technologies feeding into each other to work out hidden layers (and some of these wouldn't be initial layers) for say:

Working out where the edges are
Diameter
Language
Feature 1
Feature 2
Approximate colour

Which would feed to more hidden layers to try to give you some sort of meaningful output out of hundreds of thousands of possibilities.

Teaching the ANN to do all that would require incredible skill. Would be awesome if somebody here could do it, and make them an instant billionaire.

I'm rambling, and there's probably a way to do it less accurately.

I know NOTHING about this, but it seems that you could start with finding text, converting it to plain text, and searching for that. If you don't get anything, start inserting *s for wildcards until you find something. Then just look at your search results to see if anything is close. It doesn't seem like you would have to search through millions of records.

Quote: "oggy"​I don't see how this is doable without a huge multidisciplinary team and budget. (Note that almost all the research focuses on very small numbers of outputs)
​
​First and foremost you'd need an ANN that can scour millions and millions of images. Most of the production quality work has been done for specific coin sets and machines, to learn to identify just 1-20 coins. You'd need it to identify so many not only would you have to train it, you'd have to train it to train different iterations.
​
​Tie the ANN into some of the other methods learned in the latest article you listed and it could work, but you'd have to feed the results of that data back into the ANN too for it to properly work.
​
​It'd be an interesting project on an enterprise level, but it's the kind of thing somebody would make billions if they got it right. Think of the applications, from facial recognition to driverless cars to predicting stocks.
​
​Consider that entire fintech companies dedicate divisions to doing this, and they work with way less hidden layers than you'd need.
​
​Let's say somebody uploads a random round coin.
​
​That's your input layer
​
​You would use some of your technologies feeding into each other to work out hidden layers (and some of these wouldn't be initial layers) for say:
​
​Working out where the edges are
​Diameter
​Language
​Feature 1
​Feature 2
​Approximate colour
​
​Which would feed to more hidden layers to try to give you some sort of meaningful output out of hundreds of thousands of possibilities.
​
​Teaching the ANN to do all that would require incredible skill. Would be awesome if somebody here could do it, and make them an instant billionaire.
​
​I'm rambling, and there's probably a way to do it less accurately.

​You're absolutely right! It's a serious enterprise, the one which requires a lot of time, effort and funds.

This would be an amazing implementation.

A new page where a person would upload the photo of the coins, then he/she would select these 'layers' and position them over the text/symbols. Then the system will try to read the text and image and search though database.

Here's what I'm talking about

I can second that idea - user input will eliminate need of advanced AI.

I agree that neural networks and AI won't work for us, as we don't have millions of pictures of each coin, so the training step is not possible.
I wonder if simple image matching without neural networks could be enough.

I still see some AI in MihajloNesic's suggestion, as you need to recognise that the part of the picture in the orange square is a crown. This would need to list all recognizable symbols and train the algorithm on them.

Reading the text could be a very good approach. Still it's challenging as the text on coins might be any alphabet and it's usually not straight and not very legible.
Sometimes it won't be enough, but for most modern coins, being able to read the text should give a good hint.

There might be different approaches to made this possible. I would definitely give a try to OpenCV.

I agree the overall idea of image recognition would be great but also a huge task.

Text recognition might not be the best strategy here, since as humans, we are already quite good at reading text on the coins and inputting it in a text box to search for the coin. Maybe it would be easier to make the text-based search more powerful. If we can't read part of the characters on a coin but input those that can be read (e.g. a word with a missing character, or a wrong character), that can help.

For image-to-image recognition methods, it would help to have a database with several pictures of the same coins, rather than only one picture for each face. Is that the case?

Quote: "vatlet"​I agree the overall idea of image recognition would be great but also a huge task.
​
​Text recognition might not be the best strategy here, since as humans, we are already quite good at reading text on the coins and inputting it in a text box to search for the coin.

​Not when you don't understand the language or know difference between Japanese and Chinese for example or Arabic and Urdo etc

Better than A.I., I suggest tapping the collective wisdom of Numista's 75,000+ users, with a system like GeekMod: https://boardgamegeek.com/geekmod#toc0