Skip to content

hover.recipes

  • hover.recipes.stable

    High-level functions to produce an interactive annotation interface.

    Stable recipes whose function signatures should almost never change in the future.

    linked_annotator(dataset, **kwargs)

    Display the dataset on a 2D map in two views, one for search and one for annotation.
    Param Type Description
    dataset SupervisableDataset the dataset to link to
    **kwargs kwargs to forward to each Bokeh figure

    Expected visual layout:

    SupervisableDataset BokehDataFinder BokehDataAnnotator
    manage data subsets search -> highlight make annotations
    Source code in hover/recipes/stable.py
    @servable(title="Linked Annotator")
    def linked_annotator(dataset, **kwargs):
        """
        ???+ note "Display the dataset on a 2D map in two views, one for search and one for annotation."
    
            | Param     | Type     | Description                          |
            | :-------- | :------- | :----------------------------------- |
            | `dataset` | `SupervisableDataset` | the dataset to link to  |
            | `**kwargs` |       | kwargs to forward to each Bokeh figure |
    
            Expected visual layout:
    
            | SupervisableDataset | BokehDataFinder     | BokehDataAnnotator |
            | :------------------ | :------------------ | :----------------- |
            | manage data subsets | search -> highlight | make annotations   |
        """
        layout, _ = _linked_annotator(dataset, **kwargs)
        return layout
    

    simple_annotator(dataset, **kwargs)

    Display the dataset with on a 2D map for annotation.
    Param Type Description
    dataset SupervisableDataset the dataset to link to
    **kwargs kwargs to forward to each Bokeh figure

    Expected visual layout:

    SupervisableDataset BokehDataAnnotator
    manage data subsets make annotations
    Source code in hover/recipes/stable.py
    @servable(title="Simple Annotator")
    def simple_annotator(dataset, **kwargs):
        """
        ???+ note "Display the dataset with on a 2D map for annotation."
    
            | Param     | Type     | Description                          |
            | :-------- | :------- | :----------------------------------- |
            | `dataset` | `SupervisableDataset` | the dataset to link to  |
            | `**kwargs` |       | kwargs to forward to each Bokeh figure |
    
            Expected visual layout:
    
            | SupervisableDataset | BokehDataAnnotator |
            | :------------------ | :----------------- |
            | manage data subsets | make annotations   |
        """
        layout, _ = _simple_annotator(dataset, **kwargs)
        return layout
    


  • hover.recipes.experimental

    High-level functions to produce an interactive annotation interface.

    Experimental recipes whose function signatures might change significantly in the future. Use with caution.

    active_learning(dataset, vectorizer, vecnet_callback, **kwargs)

    Display the dataset for annotation, putting a classification model in the loop.

    Currently works most smoothly with VectorNet.

    Param Type Description
    dataset SupervisableDataset the dataset to link to
    vectorizer callable the feature -> vector function
    vecnet_callback callable the (dataset, vectorizer) -> VecNet function
    **kwargs kwargs to forward to each Bokeh figure

    Expected visual layout:

    SupervisableDataset BokehSoftLabelExplorer BokehDataAnnotator BokehDataFinder
    manage data subsets inspect model predictions make annotations search -> highlight
    Source code in hover/recipes/experimental.py
    @servable(title="Active Learning")
    def active_learning(dataset, vectorizer, vecnet_callback, **kwargs):
        """
        ???+ note "Display the dataset for annotation, putting a classification model in the loop."
            Currently works most smoothly with `VectorNet`.
    
            | Param     | Type     | Description                          |
            | :-------- | :------- | :----------------------------------- |
            | `dataset` | `SupervisableDataset` | the dataset to link to  |
            | `vectorizer` | `callable` | the feature -> vector function  |
            | `vecnet_callback` | `callable` | the (dataset, vectorizer) -> `VecNet` function|
            | `**kwargs` |       | kwargs to forward to each Bokeh figure |
    
            Expected visual layout:
    
            | SupervisableDataset | BokehSoftLabelExplorer    | BokehDataAnnotator | BokehDataFinder     |
            | :------------------ | :------------------------ | :----------------- | :------------------ |
            | manage data subsets | inspect model predictions | make annotations   | search -> highlight |
        """
        layout, _ = _active_learning(dataset, vectorizer, vecnet_callback, **kwargs)
        return layout
    

    snorkel_crosscheck(dataset, lf_list, **kwargs)

    Display the dataset for annotation, cross-checking with labeling functions.

    Use the dev set to check labeling functions; use the labeling functions to hint at potential annotation.

    Param Type Description
    dataset SupervisableDataset the dataset to link to
    lf_list list a list of callables decorated by @hover.utils.snorkel_helper.labeling_function
    **kwargs kwargs to forward to each Bokeh figure

    Expected visual layout:

    SupervisableDataset BokehSnorkelExplorer BokehDataAnnotator
    manage data subsets inspect labeling functions make annotations
    Source code in hover/recipes/experimental.py
    @servable(title="Snorkel Crosscheck")
    def snorkel_crosscheck(dataset, lf_list, **kwargs):
        """
        ???+ note "Display the dataset for annotation, cross-checking with labeling functions."
            Use the dev set to check labeling functions; use the labeling functions to hint at potential annotation.
    
            | Param     | Type     | Description                          |
            | :-------- | :------- | :----------------------------------- |
            | `dataset` | `SupervisableDataset` | the dataset to link to  |
            | `lf_list` | `list`   | a list of callables decorated by `@hover.utils.snorkel_helper.labeling_function` |
            | `**kwargs` |       | kwargs to forward to each Bokeh figure |
    
            Expected visual layout:
    
            | SupervisableDataset | BokehSnorkelExplorer       | BokehDataAnnotator |
            | :------------------ | :------------------------- | :----------------- |
            | manage data subsets | inspect labeling functions | make annotations   |
        """
        layout, _ = _snorkel_crosscheck(dataset, lf_list, **kwargs)
        return layout
    


  • hover.recipes.subroutine

    Building blocks of high-level recipes.

    Includes the following:

    • functions for creating individual standard explorers appropriate for a dataset.

    get_explorer_class(task, feature)

    Get the right hover.core.explorer class given a task and a feature.

    Can be useful for dynamically creating explorers without knowing the feature in advance.

    Param Type Description
    task str name of the task, which can be "finder", "annotator", "margin", "softlabel", or "snorkel"
    feature str name of the main feature, which can be "text", "audio" or "image"

    Usage:

    # this creates an instance of BokehTextFinder
    explorer = get_explorer_class("finder", "text")(*args, **kwargs)
    

    Source code in hover/recipes/subroutine.py
    def get_explorer_class(task, feature):
        """
        ???+ note "Get the right `hover.core.explorer` class given a task and a feature."
    
            Can be useful for dynamically creating explorers without knowing the feature in advance.
    
            | Param     | Type  | Description                          |
            | :-------- | :---- | :----------------------------------- |
            | `task`    | `str` | name of the task, which can be `"finder"`, `"annotator"`, `"margin"`, `"softlabel"`, or `"snorkel"` |
            | `feature` | `str` | name of the main feature, which can be `"text"`, `"audio"` or `"image"` |
    
            Usage:
            ```python
            # this creates an instance of BokehTextFinder
            explorer = get_explorer_class("finder", "text")(*args, **kwargs)
            ```
        """
        assert task in EXPLORER_CATALOG, f"Invalid task: {task}"
        assert feature in EXPLORER_CATALOG[task], f"Invalid feature: {feature}"
        return EXPLORER_CATALOG[task][feature]
    

    standard_annotator(dataset, **kwargs)

    Set up a BokehDataAnnotator for a SupervisableDataset.

    The annotator has a few standard interactions with the dataset:

    • read all subsets of the dataset
    • subscribe to all updates in the dataset
    • can commit annotations through selections in the "raw" subset
    Param Type Description
    dataset SupervisableDataset the dataset to link to
    **kwargs kwargs to forward to the BokehDataAnnotator
    Source code in hover/recipes/subroutine.py
    def standard_annotator(dataset, **kwargs):
        """
        ???+ note "Set up a `BokehDataAnnotator` for a `SupervisableDataset`."
    
            The annotator has a few standard interactions with the dataset:
    
            -   read all subsets of the dataset
            -   subscribe to all updates in the dataset
            -   can commit annotations through selections in the "raw" subset
    
            | Param      | Type     | Description                          |
            | :--------- | :------- | :----------------------------------- |
            | `dataset`  | `SupervisableDataset` | the dataset to link to  |
            | `**kwargs` | | kwargs to forward to the `BokehDataAnnotator` |
        """
        # auto-detect the (main) feature to use
        feature = dataset.__class__.FEATURE_KEY
        explorer_cls = get_explorer_class("annotator", feature)
    
        # first "static" version of the plot
        subsets = explorer_cls.SUBSET_GLYPH_KWARGS.keys()
        annotator = explorer_cls.from_dataset(
            dataset,
            {_k: _k for _k in subsets},
            title="Annotator: apply labels to the selected points",
            **kwargs,
        )
        annotator.plot()
    
        # subscribe for df updates
        dataset.subscribe_update_push(annotator, {_k: _k for _k in subsets})
    
        # annotators can commit to a dataset
        dataset.subscribe_data_commit(annotator, {"raw": "raw"})
    
        # annotators by default link the selection for preview
        dataset.subscribe_selection_view(annotator, ["raw", "train", "dev", "test"])
        return annotator
    

    standard_finder(dataset, **kwargs)

    Set up a BokehDataFinder for a SupervisableDataset.

    The finder has a few standard interactions with the dataset:

    • read all subsets of the dataset
    • subscribe to all updates in the dataset
    Param Type Description
    dataset SupervisableDataset the dataset to link to
    **kwargs kwargs to forward to the BokehDataFinder
    Source code in hover/recipes/subroutine.py
    def standard_finder(dataset, **kwargs):
        """
        ???+ note "Set up a `BokehDataFinder` for a `SupervisableDataset`."
    
            The finder has a few standard interactions with the dataset:
    
            -   read all subsets of the dataset
            -   subscribe to all updates in the dataset
    
            | Param      | Type     | Description                          |
            | :--------- | :------- | :----------------------------------- |
            | `dataset`  | `SupervisableDataset` | the dataset to link to  |
            | `**kwargs` | | kwargs to forward to the `BokehDataFinder` |
        """
        # auto-detect the (main) feature to use
        feature = dataset.__class__.FEATURE_KEY
        explorer_cls = get_explorer_class("finder", feature)
    
        # first "static" version of the plot
        subsets = explorer_cls.SUBSET_GLYPH_KWARGS.keys()
        finder = explorer_cls.from_dataset(
            dataset,
            {_k: _k for _k in subsets},
            title="Finder: use the search widget for highlights",
            **kwargs,
        )
        finder.plot()
    
        # subscribe for df updates
        dataset.subscribe_update_push(finder, {_k: _k for _k in subsets})
        return finder
    

    standard_snorkel(dataset, **kwargs)

    Set up a BokehSnorkelExplorer for a SupervisableDataset.

    The snorkel explorer has a few standard interactions with the dataset:

    • read "raw" and "dev" subsets of the dataset, interpreting "dev" as "labeled"
    • subscribe to all updates in those subsets
    Param Type Description
    dataset SupervisableDataset the dataset to link to
    **kwargs kwargs to forward to the BokehSnorkelExplorer
    Source code in hover/recipes/subroutine.py
    def standard_snorkel(dataset, **kwargs):
        """
        ???+ note "Set up a `BokehSnorkelExplorer` for a `SupervisableDataset`."
    
            The snorkel explorer has a few standard interactions with the dataset:
    
            -   read "raw" and "dev" subsets of the dataset, interpreting "dev" as "labeled"
            -   subscribe to all updates in those subsets
    
            | Param      | Type     | Description                          |
            | :--------- | :------- | :----------------------------------- |
            | `dataset`  | `SupervisableDataset` | the dataset to link to  |
            | `**kwargs` | | kwargs to forward to the `BokehSnorkelExplorer` |
        """
        # auto-detect the (main) feature to use
        feature = dataset.__class__.FEATURE_KEY
        explorer_cls = get_explorer_class("snorkel", feature)
    
        # first "static" version of the plot
        snorkel = explorer_cls.from_dataset(
            dataset,
            {"raw": "raw", "dev": "labeled"},
            title="Snorkel: square for correct, x for incorrect, + for missed, o for hit; click on legends to hide or show LF",
            **kwargs,
        )
        snorkel.plot()
    
        # subscribe to dataset widgets
        dataset.subscribe_update_push(snorkel, {"raw": "raw", "dev": "labeled"})
        return snorkel
    

    standard_softlabel(dataset, **kwargs)

    Set up a BokehSoftLabelExplorer for a SupervisableDataset.

    The soft label explorer has a few standard interactions with the dataset:

    • read all subsets of the dataset
    • subscribe to all updates in the dataset
    Param Type Description
    dataset SupervisableDataset the dataset to link to
    **kwargs kwargs to forward to the BokehSoftLabelExplorer
    Source code in hover/recipes/subroutine.py
    def standard_softlabel(dataset, **kwargs):
        """
        ???+ note "Set up a `BokehSoftLabelExplorer` for a `SupervisableDataset`."
    
            The soft label explorer has a few standard interactions with the dataset:
    
            -   read all subsets of the dataset
            -   subscribe to all updates in the dataset
    
            | Param      | Type     | Description                          |
            | :--------- | :------- | :----------------------------------- |
            | `dataset`  | `SupervisableDataset` | the dataset to link to  |
            | `**kwargs` | | kwargs to forward to the `BokehSoftLabelExplorer` |
        """
        # auto-detect the (main) feature to use
        feature = dataset.__class__.FEATURE_KEY
        explorer_cls = get_explorer_class("softlabel", feature)
    
        # first "static" version of the plot
        subsets = explorer_cls.SUBSET_GLYPH_KWARGS.keys()
        softlabel = explorer_cls.from_dataset(
            dataset,
            {_k: _k for _k in subsets},
            "pred_label",
            "pred_score",
            title="SoftLabel: inspect predictions and scores",
            **kwargs,
        )
        softlabel.plot()
    
        # subscribe to dataset widgets
        dataset.subscribe_update_push(softlabel, {_k: _k for _k in subsets})
        return softlabel