Feature Store Notebook¶

Feature Service¶

Fetching a Feature Service from an AmoraModel

In [23]:

  Copied!     
 
from amora.feature_store.registry import get_feature_service
from examples.amora_project.models.step_count_by_source import StepCountBySource


feature_service = get_feature_service(StepCountBySource)
feature_service
from amora.feature_store.registry import get_feature_service from examples.amora_project.models.step_count_by_source import StepCountBySource feature_service = get_feature_service(StepCountBySource) feature_service

Out[23]:

<FeatureService(_name = amora_fs__step_count_by_source, _feature_view_projections = [FeatureViewProjection(name='step_count_by_source', name_alias=None, features=[value_avg-ValueType.FLOAT, value_sum-ValueType.FLOAT, value_count-ValueType.FLOAT], join_key_map={})], _tags = {}, _description = None, _created_timestamp = None, _last_updated_timestamp = None)>

Retrieving Online features¶

Given a list of entities, return the latest available features values

Using a Feature Service reference¶

In [25]:

  Copied!     
 
from amora.feature_store import fs

features = fs.get_online_features(
    features=feature_service,
    entity_rows=[
        {"source_name": "Mi Fit"},
        {"source_name": "Diogo iPhone"},
        {"source_name": "An invalid source"}
    ]
)

features.to_df()
from amora.feature_store import fs features = fs.get_online_features( features=feature_service, entity_rows=[ {"source_name": "Mi Fit"}, {"source_name": "Diogo iPhone"}, {"source_name": "An invalid source"} ] ) features.to_df()

Out[25]:

	source_name	value_count	value_sum	value_avg
0	Mi Fit	6.0	809.0	134.833328
1	Diogo iPhone	2.0	17.0	8.500000
2	An invalid source	NaN	NaN	NaN

Using literal Feature Reference names¶

Feature reference names are generated with the format {feature_service_name}:{feature_name}

In [26]:

  Copied!     
 
from amora.feature_store import fs


features = fs.get_online_features(
    features=[
        "step_count_by_source:value_sum"
    ],
    entity_rows=[
        {"source_name": "Mi Fit"},
        {"source_name": "Diogo iPhone"},
        {"source_name": "An invalid source"}
    ]
)

features.to_df()
from amora.feature_store import fs features = fs.get_online_features( features=[ "step_count_by_source:value_sum" ], entity_rows=[ {"source_name": "Mi Fit"}, {"source_name": "Diogo iPhone"}, {"source_name": "An invalid source"} ] ) features.to_df()

Out[26]:

	source_name	value_sum
0	Mi Fit	809.0
1	Diogo iPhone	17.0
2	An invalid source	NaN

Retrieving Offline/Historical features¶

In [38]:

  Copied!     
 
from datetime import datetime, timedelta
import pandas as pd


df = pd.DataFrame(
    data={
        "source_name": [
            "Mi Fit",
            "Mi Fit",
            "Diogo iPhone",
            "Diogo iPhone",
            "iPhone",
        ],
        "event_timestamp": [
            datetime.now() - timedelta(days=30*10),
            datetime.now() - timedelta(days=30*24),
            datetime.now() - timedelta(days=30*14),
            datetime.now() - timedelta(days=30*24),
            datetime.now() - timedelta(days=30*20),
        ]
    }
)

features = fs.get_historical_features(
    features=feature_service,
    entity_df=df
)
features.to_df()
from datetime import datetime, timedelta import pandas as pd df = pd.DataFrame( data={ "source_name": [ "Mi Fit", "Mi Fit", "Diogo iPhone", "Diogo iPhone", "iPhone", ], "event_timestamp": [ datetime.now() - timedelta(days=30*10), datetime.now() - timedelta(days=30*24), datetime.now() - timedelta(days=30*14), datetime.now() - timedelta(days=30*24), datetime.now() - timedelta(days=30*20), ] } ) features = fs.get_historical_features( features=feature_service, entity_df=df ) features.to_df()

Out[38]:

	source_name	event_timestamp	value_avg	value_sum	value_count
0	Mi Fit	2021-07-01 17:54:41.332408+00:00	151.666667	910.0	6.0
1	Mi Fit	2020-05-07 17:54:41.332421+00:00	NaN	NaN	NaN
2	Diogo iPhone	2021-03-03 17:54:41.332423+00:00	197.333333	592.0	3.0
3	Diogo iPhone	2020-05-07 17:54:41.332425+00:00	NaN	NaN	NaN
4	iPhone	2020-09-04 17:54:41.332426+00:00	NaN	NaN	NaN

In [45]:

  Copied!     
 
fs.list_feature_views()
fs.list_feature_views()

Out[45]:

[<FeatureView(_name = step_count_by_source, _features = [value_avg-ValueType.FLOAT, value_sum-ValueType.FLOAT, value_count-ValueType.FLOAT], _projection = FeatureViewProjection(name='step_count_by_source', name_alias=None, features=[value_avg-ValueType.FLOAT, value_sum-ValueType.FLOAT, value_count-ValueType.FLOAT], join_key_map={}), created_timestamp = 2022-04-27 19:42:32.006933, last_updated_timestamp = 2022-04-27 20:23:03.377666, entities = ['source_name'], tags = {}, ttl = 1:00:00, online = True, input = <feast.infra.offline_stores.bigquery_source.BigQuerySource object at 0x12792eac0>, batch_source = <feast.infra.offline_stores.bigquery_source.BigQuerySource object at 0x12792eac0>, stream_source = None, materialization_intervals = [(datetime.datetime(2020, 1, 1, 0, 0, tzinfo=<UTC>), datetime.datetime(2022, 1, 1, 0, 0, tzinfo=<UTC>))])>]

HTTP Feature Serving¶

Make sure to run the feature server using amora feature-store serve.

On the examples bellow, I'm assuming a feature server accessible on localhost:8666

Listing feature views¶

In [10]:

  Copied!     
 
import requests


response = requests.get("http://localhost:8666/list-feature-views")
response.json()
import requests response = requests.get("http://localhost:8666/list-feature-views") response.json()

Out[10]:

[{'name': 'step_count_by_source',
  'features': ['step_count_by_source:value_avg',
   'step_count_by_source:value_sum',
   'step_count_by_source:value_count'],
  'entities': ['source_name']}]

Retrieving Online Feature values¶

In [18]:

  Copied!     
 
response = requests.post("http://localhost:8666/get-online-features", json={
  "features": [
    "step_count_by_source:value_avg",
    "step_count_by_source:value_sum",
    "step_count_by_source:value_count"
  ],
  "entities": {
    "source_name": [
      "Mi Fit",
      "Diogo iPhone",
      "An invalid source"
    ]
  }
})
result = response.json()
result
response = requests.post("http://localhost:8666/get-online-features", json={ "features": [ "step_count_by_source:value_avg", "step_count_by_source:value_sum", "step_count_by_source:value_count" ], "entities": { "source_name": [ "Mi Fit", "Diogo iPhone", "An invalid source" ] } }) result = response.json() result

Out[18]:

{'metadata': {'feature_names': ['source_name',
   'value_count',
   'value_sum',
   'value_avg']},
 'results': [{'values': ['Mi Fit', 6.0, 809.0, 134.8333282470703],
   'statuses': ['PRESENT', 'PRESENT', 'PRESENT', 'PRESENT'],
   'event_timestamps': ['1970-01-01T00:00:00Z',
    '2021-07-23T02:00:00Z',
    '2021-07-23T02:00:00Z',
    '2021-07-23T02:00:00Z']},
  {'values': ['Diogo iPhone', 2.0, 17.0, 8.5],
   'statuses': ['PRESENT', 'PRESENT', 'PRESENT', 'PRESENT'],
   'event_timestamps': ['1970-01-01T00:00:00Z',
    '2021-07-23T02:00:00Z',
    '2021-07-23T02:00:00Z',
    '2021-07-23T02:00:00Z']},
  {'values': ['An invalid source', None, None, None],
   'statuses': ['PRESENT', 'NOT_FOUND', 'NOT_FOUND', 'NOT_FOUND'],
   'event_timestamps': ['1970-01-01T00:00:00Z',
    '2021-07-23T02:00:00Z',
    '2021-07-23T02:00:00Z',
    '2021-07-23T02:00:00Z']}]}

As a `pandas.DataFrame`¶

In [17]:

  Copied!     
 
import pandas as pd


pd.DataFrame(
    columns=result['metadata']['feature_names'],
    data=[r['values'] for r in result['results']]
)
import pandas as pd pd.DataFrame( columns=result['metadata']['feature_names'], data=[r['values'] for r in result['results']] )

Out[17]:

	source_name	value_count	value_sum	value_avg
0	Mi Fit	6.0	809.0	134.833328
1	Diogo iPhone	2.0	17.0	8.500000
2	An invalid source	NaN	NaN	NaN

Last update: 2023-11-23