Comments on: On Building a Stupidly Fast Graph Database http://blog.directededge.com/2009/02/27/on-building-a-stupidly-fast-graph-database/ Sun, 31 Jan 2010 19:10:39 -0700 http://wordpress.org/?v=2.9.2 hourly 1 By: Yves http://blog.directededge.com/2009/02/27/on-building-a-stupidly-fast-graph-database/comment-page-1/#comment-4645 Yves Fri, 07 Aug 2009 18:57:07 +0000 http://blog.directededge.com/?p=175#comment-4645 Simply amazing. Simply amazing.

]]> By: Trabayo » Blog Archive » Directed Edge auf Techcrunch http://blog.directededge.com/2009/02/27/on-building-a-stupidly-fast-graph-database/comment-page-1/#comment-4644 Trabayo » Blog Archive » Directed Edge auf Techcrunch Fri, 07 Aug 2009 13:17:02 +0000 http://blog.directededge.com/?p=175#comment-4644 [...] Ähnlichkeiten und Verbindungen zwischen Elementen. Und das ziemlich schnell: Sie haben dafür eine eigene Datenbank entwickelt. Weil alle anderen einfach nicht schnell genug [...] [...] Ähnlichkeiten und Verbindungen zwischen Elementen. Und das ziemlich schnell: Sie haben dafür eine eigene Datenbank entwickelt. Weil alle anderen einfach nicht schnell genug [...]

]]> By: Webは粗放な「検索」から精度の良い「推奨」に http://blog.directededge.com/2009/02/27/on-building-a-stupidly-fast-graph-database/comment-page-1/#comment-4643 Webは粗放な「検索」から精度の良い「推奨」に Fri, 07 Aug 2009 04:27:26 +0000 http://blog.directededge.com/?p=175#comment-4643 [...] Wheelerによれば、それが可能なのは自家製のグラフ型データベースを使うからで、既存の商用データベース〔多くが関係データベース〕では必要な速度が得られない。Directed EdgeもLindenのように、Webは検索から推奨に移行すると信じている。そしてそのためにはリアルタイム性が必須だ。リアルタイムWebは今ソーシャルの方面で離陸したばかりだが、やがてWeb全体に広まるとWheelerは考えている。”その移行はすでに起きつつあると思う。うちは、その大波に乗るとともに大波の一つでありたい“、とWheelerは言っている。 [...] [...] Wheelerによれば、それが可能なのは自家製のグラフ型データベースを使うからで、既存の商用データベース〔多くが関係データベース〕では必要な速度が得られない。Directed EdgeもLindenのように、Webは検索から推奨に移行すると信じている。そしてそのためにはリアルタイム性が必須だ。リアルタイムWebは今ソーシャルの方面で離陸したばかりだが、やがてWeb全体に広まるとWheelerは考えている。”その移行はすでに起きつつあると思う。うちは、その大波に乗るとともに大波の一つでありたい“、とWheelerは言っている。 [...]

]]> By: YC-Funded Directed Edge Sees A Post-Search Web Where Recommendations Rule http://blog.directededge.com/2009/02/27/on-building-a-stupidly-fast-graph-database/comment-page-1/#comment-4642 YC-Funded Directed Edge Sees A Post-Search Web Where Recommendations Rule Thu, 06 Aug 2009 09:11:36 +0000 http://blog.directededge.com/?p=175#comment-4642 [...] of the graph database they created in-house after they realized the off-the-shelf options just weren’t good enough for what they wanted to do. And much like Linden’s quote, Directed Edge truly believes that [...] [...] of the graph database they created in-house after they realized the off-the-shelf options just weren’t good enough for what they wanted to do. And much like Linden’s quote, Directed Edge truly believes that [...]

]]> By: Scott Wheeler http://blog.directededge.com/2009/02/27/on-building-a-stupidly-fast-graph-database/comment-page-1/#comment-4641 Scott Wheeler Mon, 03 Aug 2009 17:01:06 +0000 http://blog.directededge.com/?p=175#comment-4641 @didroe – It’s something I’ve thought of, but there’s no scheme for it yet. In practice hot blocks end up being in disk cache, so the primary advantage would be in having a higher percentage of the actual working set in the set of cached disk pages. Unfortunately the working set being used for recommendations doesn’t necessarily correlate to the simple link structure, so an optimization would probably do better to reorder blocks based on measured access patterns. @didroe – It’s something I’ve thought of, but there’s no scheme for it yet. In practice hot blocks end up being in disk cache, so the primary advantage would be in having a higher percentage of the actual working set in the set of cached disk pages. Unfortunately the working set being used for recommendations doesn’t necessarily correlate to the simple link structure, so an optimization would probably do better to reorder blocks based on measured access patterns.

]]> By: didroe http://blog.directededge.com/2009/02/27/on-building-a-stupidly-fast-graph-database/comment-page-1/#comment-4640 didroe Mon, 03 Aug 2009 16:42:17 +0000 http://blog.directededge.com/?p=175#comment-4640 Do you worry about data locality? It seems to me that it would make a big difference if the things you are linking to (sinks) are in the same data block as the things that link to them (sources). Do you have a scheme for that or is that not really an issue? Do you worry about data locality? It seems to me that it would make a big difference if the things you are linking to (sinks) are in the same data block as the things that link to them (sources). Do you have a scheme for that or is that not really an issue?

]]> By: spenser http://blog.directededge.com/2009/02/27/on-building-a-stupidly-fast-graph-database/comment-page-1/#comment-4637 spenser Sat, 27 Jun 2009 09:32:42 +0000 http://blog.directededge.com/?p=175#comment-4637 Nice read. Now I have to go find where I stashed my copy of this thing. Yep, did just about this exact thing in 2003 or so. This was just the reminder I needed. :D Nice read.

Now I have to go find where I stashed my copy of this thing.

Yep, did just about this exact thing in 2003 or so. This was just the reminder I needed. :D

]]> By: Nicolas http://blog.directededge.com/2009/02/27/on-building-a-stupidly-fast-graph-database/comment-page-1/#comment-4638 Nicolas Thu, 25 Jun 2009 17:35:53 +0000 http://blog.directededge.com/?p=175#comment-4638 Hey Great article. At which level do you deal with users rights and authorizations ? And is this consistent with security matters ? Hey
Great article. At which level do you deal with users rights and authorizations ? And is this consistent with security matters ?

]]> By: Should you go Beyond Relational Databases? | Think Vitamin http://blog.directededge.com/2009/02/27/on-building-a-stupidly-fast-graph-database/comment-page-1/#comment-4639 Should you go Beyond Relational Databases? | Think Vitamin Wed, 24 Jun 2009 21:17:16 +0000 http://blog.directededge.com/?p=175#comment-4639 [...] (which typically uses MySQL or PostgreSQL as storage back-end) are ones to look at. FreeBase and DirectedEdge have developed graph databases for their internal [...] [...] (which typically uses MySQL or PostgreSQL as storage back-end) are ones to look at. FreeBase and DirectedEdge have developed graph databases for their internal [...]

]]> By: Nathan Kurz http://blog.directededge.com/2009/02/27/on-building-a-stupidly-fast-graph-database/comment-page-1/#comment-4633 Nathan Kurz Tue, 07 Apr 2009 01:45:53 +0000 http://blog.directededge.com/?p=175#comment-4633 Hi Scott -- I appreciate the article. I read it when it came out, and have thought about it since. I'm very familiar with mmap(), and have written a simpler version of such a data store myself, but I'm still not understanding your locking scheme. You say you swap the data pointers when a write lock comes in. How do you guarantee that a reader is not still reading at some offset into the vector? Wouldn't it be simpler to allocate new space for the writer, instead of switching where the reader is supposed to look? Hi Scott –

I appreciate the article. I read it when it came out, and have thought about it since. I’m very familiar with mmap(), and have written a simpler version of such a data store myself, but I’m still not understanding your locking scheme.

You say you swap the data pointers when a write lock comes in. How do you guarantee that a reader is not still reading at some offset into the vector? Wouldn’t it be simpler to allocate new space for the writer, instead of switching where the reader is supposed to look?

]]>