Discussion:
N-dimensional Point Indexing
Luís Filipe Nassif
2018-02-06 16:17:33 UTC
Permalink
Hi all,

Lucene is able to index generic n-dimensional points for efficient
similarity or nearest neightbors search? I have looked at spatial package
in the past but seems it is specific to geo points? The use case is to
index image feature vectors to search for similar images in a corpus.

Currently we are using lucene to text search and we would like to not have
to manage two different index structures, synchronize commits, so on.

Thank you,
Luis Nassif
Luís Filipe Nassif
2018-02-06 17:35:17 UTC
Permalink
Sorry, I was looking at the wrong place. Should I use BinaryPoint (
https://lucene.apache.org/core/6_0_0/core/org/apache/lucene/document/BinaryPoint.html)
?
Post by Luís Filipe Nassif
Hi all,
Lucene is able to index generic n-dimensional points for efficient
similarity or nearest neightbors search? I have looked at spatial package
in the past but seems it is specific to geo points? The use case is to
index image feature vectors to search for similar images in a corpus.
Currently we are using lucene to text search and we would like to not have
to manage two different index structures, synchronize commits, so on.
Thank you,
Luis Nassif
Luís Filipe Nassif
2018-02-06 18:07:24 UTC
Permalink
Is it limited up to 8 dimensions as described at
https://www.elastic.co/blog/lucene-points-6.0?
Post by Luís Filipe Nassif
Sorry, I was looking at the wrong place. Should I use BinaryPoint (
https://lucene.apache.org/core/6_0_0/core/org/apache/
lucene/document/BinaryPoint.html) ?
Post by Luís Filipe Nassif
Hi all,
Lucene is able to index generic n-dimensional points for efficient
similarity or nearest neightbors search? I have looked at spatial package
in the past but seems it is specific to geo points? The use case is to
index image feature vectors to search for similar images in a corpus.
Currently we are using lucene to text search and we would like to not
have to manage two different index structures, synchronize commits, so on.
Thank you,
Luis Nassif
Luís Filipe Nassif
2018-02-27 00:03:44 UTC
Permalink
Hi Lucene community,

Is BinaryPoint limited up to 8 dimensions?

Thanks,
Luis

Em 6 de fev de 2018 16:07, "Luís Filipe Nassif" <***@gmail.com>
escreveu:

Is it limited up to 8 dimensions as described at
https://www.elastic.co/blog/lucene-points-6.0?
Post by Luís Filipe Nassif
Sorry, I was looking at the wrong place. Should I use BinaryPoint (
https://lucene.apache.org/core/6_0_0/core/org/apache/lucene
/document/BinaryPoint.html) ?
Post by Luís Filipe Nassif
Hi all,
Lucene is able to index generic n-dimensional points for efficient
similarity or nearest neightbors search? I have looked at spatial package
in the past but seems it is specific to geo points? The use case is to
index image feature vectors to search for similar images in a corpus.
Currently we are using lucene to text search and we would like to not
have to manage two different index structures, synchronize commits, so on.
Thank you,
Luis Nassif
Adrien Grand
2018-02-27 00:19:16 UTC
Permalink
Yes it is.
Post by Luís Filipe Nassif
Hi Lucene community,
Is BinaryPoint limited up to 8 dimensions?
Thanks,
Luis
Is it limited up to 8 dimensions as described at
https://www.elastic.co/blog/lucene-points-6.0?
Post by Luís Filipe Nassif
Sorry, I was looking at the wrong place. Should I use BinaryPoint (
https://lucene.apache.org/core/6_0_0/core/org/apache/lucene
/document/BinaryPoint.html) ?
Post by Luís Filipe Nassif
Hi all,
Lucene is able to index generic n-dimensional points for efficient
similarity or nearest neightbors search? I have looked at spatial
package
Post by Luís Filipe Nassif
Post by Luís Filipe Nassif
in the past but seems it is specific to geo points? The use case is to
index image feature vectors to search for similar images in a corpus.
Currently we are using lucene to text search and we would like to not
have to manage two different index structures, synchronize commits, so
on.
Post by Luís Filipe Nassif
Post by Luís Filipe Nassif
Thank you,
Luis Nassif
Luís Filipe Nassif
2018-02-27 00:24:00 UTC
Permalink
Thank you, Adrian.
Post by Adrien Grand
Yes it is.
Post by Luís Filipe Nassif
Hi Lucene community,
Is BinaryPoint limited up to 8 dimensions?
Thanks,
Luis
Is it limited up to 8 dimensions as described at
https://www.elastic.co/blog/lucene-points-6.0?
Post by Luís Filipe Nassif
Sorry, I was looking at the wrong place. Should I use BinaryPoint (
https://lucene.apache.org/core/6_0_0/core/org/apache/lucene
/document/BinaryPoint.html) ?
Post by Luís Filipe Nassif
Hi all,
Lucene is able to index generic n-dimensional points for efficient
similarity or nearest neightbors search? I have looked at spatial
package
Post by Luís Filipe Nassif
Post by Luís Filipe Nassif
in the past but seems it is specific to geo points? The use case is to
index image feature vectors to search for similar images in a corpus.
Currently we are using lucene to text search and we would like to not
have to manage two different index structures, synchronize commits, so
on.
Post by Luís Filipe Nassif
Post by Luís Filipe Nassif
Thank you,
Luis Nassif
Ken Krugler
2018-10-18 00:32:24 UTC
Permalink
I’ve been looking at directly storing feature vectors and providing scoring/filtering support.

This is for vectors consisting of (typically 300 - 2048) floats or doubles.

It’s following the same pattern as geospatial support - so a new field type and query/parser, plus plumbing to hook it into Solr.

Before I go much further, is there anything like this already done, or in the works?

Thanks,

— Ken
Post by Luís Filipe Nassif
Thank you, Adrian.
Post by Adrien Grand
Yes it is.
Post by Luís Filipe Nassif
Hi Lucene community,
Is BinaryPoint limited up to 8 dimensions?
Thanks,
Luis
Is it limited up to 8 dimensions as described at
https://www.elastic.co/blog/lucene-points-6.0?
Post by Luís Filipe Nassif
Sorry, I was looking at the wrong place. Should I use BinaryPoint (
https://lucene.apache.org/core/6_0_0/core/org/apache/lucene
/document/BinaryPoint.html) ?
Post by Luís Filipe Nassif
Hi all,
Lucene is able to index generic n-dimensional points for efficient
similarity or nearest neightbors search? I have looked at spatial
package
Post by Luís Filipe Nassif
Post by Luís Filipe Nassif
in the past but seems it is specific to geo points? The use case is to
index image feature vectors to search for similar images in a corpus.
Currently we are using lucene to text search and we would like to not
have to manage two different index structures, synchronize commits, so
on.
Post by Luís Filipe Nassif
Post by Luís Filipe Nassif
Thank you,
Luis Nassif
--------------------------
Ken Krugler
+1 530-210-6378
http://www.scaleunlimited.com
Custom big data solutions & training
Flink, Solr, Hadoop, Cascading & Cassandra
Adrien Grand
2018-11-14 14:07:19 UTC
Permalink
If you need them for scoring, then the natural choice would be to
encode them in a BinaryDocValuesField. How do you plan to filter on
these filter vectors? This is too many dimensions for points and doc
values are not good at filtering.
I’ve been looking at directly storing feature vectors and providing scoring/filtering support.
This is for vectors consisting of (typically 300 - 2048) floats or doubles.
It’s following the same pattern as geospatial support - so a new field type and query/parser, plus plumbing to hook it into Solr.
Before I go much further, is there anything like this already done, or in the works?
Thanks,
— Ken
Post by Luís Filipe Nassif
Thank you, Adrian.
Post by Adrien Grand
Yes it is.
Post by Luís Filipe Nassif
Hi Lucene community,
Is BinaryPoint limited up to 8 dimensions?
Thanks,
Luis
Is it limited up to 8 dimensions as described at
https://www.elastic.co/blog/lucene-points-6.0?
Post by Luís Filipe Nassif
Sorry, I was looking at the wrong place. Should I use BinaryPoint (
https://lucene.apache.org/core/6_0_0/core/org/apache/lucene
/document/BinaryPoint.html) ?
Post by Luís Filipe Nassif
Hi all,
Lucene is able to index generic n-dimensional points for efficient
similarity or nearest neightbors search? I have looked at spatial
package
Post by Luís Filipe Nassif
Post by Luís Filipe Nassif
in the past but seems it is specific to geo points? The use case is to
index image feature vectors to search for similar images in a corpus.
Currently we are using lucene to text search and we would like to not
have to manage two different index structures, synchronize commits, so
on.
Post by Luís Filipe Nassif
Post by Luís Filipe Nassif
Thank you,
Luis Nassif
--------------------------
Ken Krugler
+1 530-210-6378
http://www.scaleunlimited.com
Custom big data solutions & training
Flink, Solr, Hadoop, Cascading & Cassandra
--
Adrien

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-***@lucene.apache.org
For additional commands, e-mail: java-user-***@lucene.apache.org
Loading...