Skip to main content

System Architecture


Processing a query

Fetch data from Upstreams

When performing a data fetching process, RelationService will

  1. Iter all available upstreams which can handle this query
  2. Request and collect results from each upstream available
  3. Newly fetched data will be fed into this circulation again, until no new data is yield.

A psuedo code snippet describing this:

up_next = [{platform: :platform_to_query, identity: "identity_to_query"}]
fetched = []

until up_next.empty? do
processing = up_next.pop()
results = fetch_one(processing)
results.each do |result|
next if up_next.contains?(result) || fetched.contains?(result)

# Query all available upstream for connections of given identity.
def fetch_one(identity) do |upstream|
upstream.can_handle?(identity) ? upstream.perform_query(identity) : []

Actual code can be found at src/upstream/ -> pub async fn fetch_all() definition

So you may found a "code search" kind of slow in the first time, but when it is fetched and cached in RelationService DB, second query will become pretty fast.

See also