Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat(v2): rich display for doc and da #1043

Merged
merged 46 commits into from Jan 25, 2023
Merged

feat(v2): rich display for doc and da #1043

merged 46 commits into from Jan 25, 2023

Conversation

anna-charlotte
Copy link
Contributor

@anna-charlotte anna-charlotte commented Jan 19, 2023 •
edited

Goals:

  • Add .summary() to for Document and DocumentArray.
  • Add pretty print for Document as well as DocumentArray.

Pretty prints:
image

For Document:

  • doc.summary() to display the Document instance
    • key, value (including type and shape)
    • maybe display shorter id: 2184b5 ... instead of 2184b53f977f566d72f72a6e706edb00
    • if eg. list of 200 elements is part of the Document, don't show all 200 elements, keep max size of such display manageable.
      Example:

image

  • doc.schema_summary() to display the Document's schema

For DocumentArray:

  • schema and length
  • redundant fields for v2:
    • Homogenous Documents
    • Common Attributes, instead show the schema
    • Multimodal dataclass

image

@anna-charlotte anna-charlotte linked an issue Jan 19, 2023 that may be closed by this pull request
V2: rich display of Document and DocumentArray #1025
Closed
@samsja
Copy link
Member

samsja commented Jan 20, 2023

@anna-charlotte I believe we should tackle pretty printing in this PR as well.

at the moment pritting a document array will lool like this <docarray.array.abstract_array.DocumentArray[Document][MyDoc] at 0x7fb93800b5e0> which is not very informative. Same apply to document

@anna-charlotte
Copy link
Contributor Author

Wrt pretty prints: this PR only includes the pretty print for DocumentArray but not for Document, since for the latter I encountered the following problem:

rich provides a nice pretty print for pydantic models, which we would like to use as the Documents __repr__().

class MyDoc(pydantic.BaseModel):
    title: str

mydoc = MyDoc(title='my title')
rich.print(mydoc)

output:

MyDoc(
    title='my title'
)

But for the Documents .summary() we overwrite the __rich_console__() function to the tabular print wanted for the summary such as:

mydoc.summary()
-----------------------------------------
| Attribute            | Value          |
-----------------------------------------
| title                | 'my title'     |
-----------------------------------------

This __rich_console__() is also being called in the rich.print(mydoc), which results in a tabular summary display for the pretty print instead of the pydantic pretty print.

I opened a question in their discussion to see if there is a nice way to solve this.

Therefore for now only pretty print for DocumenArray.

samsja
samsja reviewed Jan 24, 2023
View reviewed changes
docarray/math/helper.py Outdated Show resolved Hide resolved
@github-actions github-actions bot added size/l and removed size/m labels Jan 24, 2023
@anna-charlotte anna-charlotte marked this pull request as ready for review January 24, 2023 13:43
@JohannesMessner
Copy link
Member

@anna-charlotte could you please link this PR/Issue to the dev roadmap and tick it off once merged?

JohannesMessner
JohannesMessner approved these changes Jan 24, 2023
View reviewed changes
Copy link
Member

@JohannesMessner JohannesMessner left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Honestly I am lost in all of this rich stuff, but the screenshots look pretty, so that's a thumbs up from me ^^

@anna-charlotte
fix: remove redundant line
58229aa 
Signed-off-by: anna-charlotte <charlotte.gerhaher@jina.ai>
@anna-charlotte
fix: remove comments
e55ba3b 
Signed-off-by: anna-charlotte <charlotte.gerhaher@jina.ai>
@anna-charlotte
feat: add schema highlighter
147742d 
Signed-off-by: anna-charlotte <charlotte.gerhaher@jina.ai>
@anna-charlotte
fix: add plotmixin to mixin init
59bd3a6 
Signed-off-by: anna-charlotte <charlotte.gerhaher@jina.ai>
@anna-charlotte
fix: adjust da summary
fd26a43 
Signed-off-by: anna-charlotte <charlotte.gerhaher@jina.ai>
@anna-charlotte
fix: move minmaxnormalize to comp backend
675b5c5 
Signed-off-by: anna-charlotte <charlotte.gerhaher@jina.ai>
@anna-charlotte
fix: remove redundant lines
a375d19 
Signed-off-by: anna-charlotte <charlotte.gerhaher@jina.ai>
@anna-charlotte
fix: add squeeze and detach to comp backend
c3b44bd 
Signed-off-by: anna-charlotte <charlotte.gerhaher@jina.ai>
@anna-charlotte
fix: apply suggestion from code review
0d5653c 
Signed-off-by: anna-charlotte <charlotte.gerhaher@jina.ai>
@anna-charlotte
refactor: rename iterable attrs
6d479ab 
Signed-off-by: anna-charlotte <charlotte.gerhaher@jina.ai>
@anna-charlotte
fix: clean up
a1c4678 
Signed-off-by: anna-charlotte <charlotte.gerhaher@jina.ai>
@anna-charlotte
fix: import
3aac1c9 
Signed-off-by: anna-charlotte <charlotte.gerhaher@jina.ai>
@anna-charlotte
fix: iterate over fields instead of annotations
eb75060 
Signed-off-by: anna-charlotte <charlotte.gerhaher@jina.ai>
@anna-charlotte
fix: remove math package since moved to comp backends
3cc1b55 
Signed-off-by: anna-charlotte <charlotte.gerhaher@jina.ai>
@anna-charlotte
refactor: use single quotes
ab585eb 
Signed-off-by: anna-charlotte <charlotte.gerhaher@jina.ai>
@anna-charlotte
fix: apply suggestions from code review
b838ec9 
Signed-off-by: anna-charlotte <charlotte.gerhaher@jina.ai>
@anna-charlotte
fix: extract summary to doc summary class
c56aa6e 
Signed-off-by: anna-charlotte <charlotte.gerhaher@jina.ai>
@anna-charlotte
fix: add pretty print for base document
b2b5bdd 
Signed-off-by: anna-charlotte <charlotte.gerhaher@jina.ai>
@anna-charlotte
fix: use rich capture instead of string io
7aa7e58 
Signed-off-by: anna-charlotte <charlotte.gerhaher@jina.ai>
@anna-charlotte
fix: add colors for optional and union and use only single quotes
2ae8d6a 
Signed-off-by: anna-charlotte <charlotte.gerhaher@jina.ai>
@anna-charlotte
fix: extract display classes to display package
0b881b1 
Signed-off-by: anna-charlotte <charlotte.gerhaher@jina.ai>
@anna-charlotte
fix: make da not optional in da summary
6ba4eff 
Signed-off-by: anna-charlotte <charlotte.gerhaher@jina.ai>
@anna-charlotte
fix: set _console instead of initializing new one everytime in __str__
a70142a 
Signed-off-by: anna-charlotte <charlotte.gerhaher@jina.ai>
@anna-charlotte
fix: put console at module level
2a6bd5c 
Signed-off-by: anna-charlotte <charlotte.gerhaher@jina.ai>
@github-actions
Copy link

📝 Docs are deployed on https://ft-feat-rich-display--jina-docs.netlify.app 🎉

@samsja samsja merged commit ac28cf3 into feat-rewrite-v2 Jan 25, 2023
@samsja samsja deleted the feat-rich-display branch January 25, 2023 13:22
JoanFM pushed a commit that referenced this pull request Jan 26, 2023
@anna-charlotte @JoanFM
feat(v2): rich display for doc and da (#1043)
6d52497 
* feat: add rich display for doc and da

Signed-off-by: anna-charlotte <charlotte.gerhaher@jina.ai>

* fix: wip plot

Signed-off-by: anna-charlotte <charlotte.gerhaher@jina.ai>

* fix: wip plot

Signed-off-by: anna-charlotte <charlotte.gerhaher@jina.ai>

* fix: wip plot

Signed-off-by: anna-charlotte <charlotte.gerhaher@jina.ai>

* feat: add math package and minmax normalize

Signed-off-by: anna-charlotte <charlotte.gerhaher@jina.ai>

* fix: summary for document

Signed-off-by: anna-charlotte <charlotte.gerhaher@jina.ai>

* chore: update poetry lock after rebase

Signed-off-by: anna-charlotte <charlotte.gerhaher@jina.ai>

* fix: move all from plotmixin to base document

Signed-off-by: anna-charlotte <charlotte.gerhaher@jina.ai>

* feat: add docs schema summary

Signed-off-by: anna-charlotte <charlotte.gerhaher@jina.ai>

* feat: add document array summary

Signed-off-by: anna-charlotte <charlotte.gerhaher@jina.ai>

* fix: display doc within doc

Signed-off-by: anna-charlotte <charlotte.gerhaher@jina.ai>

* fix: in notebook print docs summary

Signed-off-by: anna-charlotte <charlotte.gerhaher@jina.ai>

* fix: move summary from da to abstract da

Signed-off-by: anna-charlotte <charlotte.gerhaher@jina.ai>

* fix: get schema for doc

Signed-off-by: anna-charlotte <charlotte.gerhaher@jina.ai>

* fix: wip doc summary

Signed-off-by: anna-charlotte <charlotte.gerhaher@jina.ai>

* fix: wip clean up

Signed-off-by: anna-charlotte <charlotte.gerhaher@jina.ai>

* test: add test for da pretty print

Signed-off-by: anna-charlotte <charlotte.gerhaher@jina.ai>

* docs: update note

Signed-off-by: anna-charlotte <charlotte.gerhaher@jina.ai>

* docs: add some documentation

Signed-off-by: anna-charlotte <charlotte.gerhaher@jina.ai>

* fix: apply samis suggestion

Signed-off-by: anna-charlotte <charlotte.gerhaher@jina.ai>

* fix: mypy checks

Signed-off-by: anna-charlotte <charlotte.gerhaher@jina.ai>

* fix: move to plot mixin

Signed-off-by: anna-charlotte <charlotte.gerhaher@jina.ai>

* fix: remove redundant line

Signed-off-by: anna-charlotte <charlotte.gerhaher@jina.ai>

* fix: remove comments

Signed-off-by: anna-charlotte <charlotte.gerhaher@jina.ai>

* feat: add schema highlighter

Signed-off-by: anna-charlotte <charlotte.gerhaher@jina.ai>

* fix: add plotmixin to mixin init

Signed-off-by: anna-charlotte <charlotte.gerhaher@jina.ai>

* fix: adjust da summary

Signed-off-by: anna-charlotte <charlotte.gerhaher@jina.ai>

* fix: move minmaxnormalize to comp backend

Signed-off-by: anna-charlotte <charlotte.gerhaher@jina.ai>

* fix: remove redundant lines

Signed-off-by: anna-charlotte <charlotte.gerhaher@jina.ai>

* fix: add squeeze and detach to comp backend

Signed-off-by: anna-charlotte <charlotte.gerhaher@jina.ai>

* fix: apply suggestion from code review

Signed-off-by: anna-charlotte <charlotte.gerhaher@jina.ai>

* refactor: rename iterable attrs

Signed-off-by: anna-charlotte <charlotte.gerhaher@jina.ai>

* fix: clean up

Signed-off-by: anna-charlotte <charlotte.gerhaher@jina.ai>

* fix: import

Signed-off-by: anna-charlotte <charlotte.gerhaher@jina.ai>

* fix: iterate over fields instead of annotations

Signed-off-by: anna-charlotte <charlotte.gerhaher@jina.ai>

* fix: remove math package since moved to comp backends

Signed-off-by: anna-charlotte <charlotte.gerhaher@jina.ai>

* refactor: use single quotes

Signed-off-by: anna-charlotte <charlotte.gerhaher@jina.ai>

* fix: apply suggestions from code review

Signed-off-by: anna-charlotte <charlotte.gerhaher@jina.ai>

* fix: extract summary to doc summary class

Signed-off-by: anna-charlotte <charlotte.gerhaher@jina.ai>

* fix: add pretty print for base document

Signed-off-by: anna-charlotte <charlotte.gerhaher@jina.ai>

* fix: use rich capture instead of string io

Signed-off-by: anna-charlotte <charlotte.gerhaher@jina.ai>

* fix: add colors for optional and union and use only single quotes

Signed-off-by: anna-charlotte <charlotte.gerhaher@jina.ai>

* fix: extract display classes to display package

Signed-off-by: anna-charlotte <charlotte.gerhaher@jina.ai>

* fix: make da not optional in da summary

Signed-off-by: anna-charlotte <charlotte.gerhaher@jina.ai>

* fix: set _console instead of initializing new one everytime in __str__

Signed-off-by: anna-charlotte <charlotte.gerhaher@jina.ai>

* fix: put console at module level

Signed-off-by: anna-charlotte <charlotte.gerhaher@jina.ai>

Signed-off-by: anna-charlotte <charlotte.gerhaher@jina.ai>
Signed-off-by: Joan Fontanals Martinez <joan.martinez@jina.ai>
@anna-charlotte anna-charlotte mentioned this pull request Feb 20, 2023
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@JohannesMessner JohannesMessner JohannesMessner approved these changes

@samsja samsja samsja approved these changes

Assignees
No one assigned
Labels
area/core area/setup area/testing component/array size/l
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

V2: rich display of Document and DocumentArray
3 participants
@anna-charlotte @samsja @JohannesMessner