Rem Cluster - a group of rems that can be reviewed together in sequential order

liberated_potato · July 29, 2021, 3:45pm

What is this?
A Rem Cluster is a small group (a single digit number) of rems that represent a complex concept. The SRS algorithm schedules all the rems in that rem cluster together and in the same order they appear in the document

Why is this needed?
I agree with the fact that it is better to review flashcards in random order. However, I think the most overlooked aspect of this randomization is ‘at what level of detail should we randomize’. If we take the case of rote memorization of vocabulary cards, it totally makes sense to randomize flashcards corresponding to different words. On the other hand, if we take complex concepts from any field that need understanding (as opposed to rote memorization) often we need multiple flashcards to completely represent that concept. If I use my learning approach as an example, I just ask few questions about a complex concept to enforce my understanding about it. In that case, if these flashcards are asked in random order during review it may not be ideal to get the correct full picture. The worse part is when these cards mature. As we do more reviews the cards mature and the interval between reviews increase. These related flashcards (or questions in my case) may end up in a state in which I get one related flash card today and another related flash card two weeks later. Thus without a proper order, spaced repetition systems fragment the the concept and fail to help us in understanding the full picture. If you are serious about learning, you probably know that this is one of the biggest draw back of SRS ( Read more A vote against spaced repetition - Flashcards are less useful to learning the “big picture”)

Related Topics

Implementation
The implementation can be as simple as introducing a Power-Up rem that can be applied to a parent. Parent tagged with this particular Power-Up will treat its children (only children, not descendants) as a Rem Cluster

What can go wrong with this idea?

Won’t this prevent reviews from getting scheduled uniformly over time?
I think if you use it correctly it won’t create a big imbalance. The idea is to create a small group (I prefer a group of not more than 5 rems). Above all, you are not doing more number of reviews, it is just that the reviews are grouped logically
What if I forget one card from a cluster?
Ideally there should be a setting so that the user can decide what to do when this happens. One option is to consider the whole cluster as forgotten, but that is not ideal. Another smarter option would be to take into account the next repetition time of the cluster and schedule reviews for the forgotten card in such a way that by the time the cluster gets reviewed next time, you would remember the forgotten card.
What else ? any thoughts/feedbacks ?

UMNiK · November 22, 2020, 7:43am

Have you tried settings-experimental-document cards?

hannesfrank · November 30, 2020, 10:50am

Related to Clozes: hide all, combine clozes, clues/hints, start/end points for Queue’s and more! (a system to fully customize Queue’s. Please read the description) and the new Extra Card Detail Power Up which both try to get more context onto a card.

There are also the Cluster Settings (Queue > Ordering > Document|Reference|Tag Clusters). Maybe this FR could be incorporated into this ordering algorithm somehow. But for that we need to know what these settings do exactly.

An easy first step would be to (when a Concept is due) to also ask all descriptors and direct child concepts. This of course clashes with the normal scheduler (e.g. it leads to more reviews = time investment if you know the context of a rem already well). So maybe we could get an extra button “Review Context/Child/?” to chose in queue if we want to review the context of a rem.

liberated_potato · February 12, 2021, 6:47am

Yea, even I don’t know if this Queue - Ordering option does anything. I just see random cards all the time

Like you said, if we ask descriptors as well while reviewing a concept it will create more reviews. So thee extra review overhead there may not be ideal. That is why I was thinking if we tag a parent (not necessarily a concept) with some Rem Cluster power up, then all of its children (only children, not descendants to avoid recursive effect) get reviewed together in sequential order. If this is the case, we are not adding any extra reviews, just logically grouping the existing reviews and that too only if the user wants. However, when you forget a card from this cluster, then it becomes tricky. One solution is what I mentioned in (2), which I think is reasonable solution.

Having a “Review Context/Child?” button will be helpful regardless of this FR. If the review GUI was bit more sophisticated, then some hints (as simple as how many children are there, or just front card of the children as a list, or front card + health/next repetition time of those child cards) about the context/children can also be presented to the user so that the user can make a better decision. But then again that may alter the review history of the descriptors and interfere with the default algorithm. Overall, I think the Remnote review system is still biased due to/influenced by the Anki style of reviewing. Remnote is capable of network/graph style review instead of this traditional Anki type singular card style reviews.

The extra card details and that Cloze FR are good stuff. I hope at some point, devs will consider revamping the review system in order to avoid fragmentation of cards (through FRs like this one) and to improve contextual clues (like hints, hierarchy, that cloze FR etc.). For the contextual clue part I am really grateful to have hierarchy in the cards and (now) the extra card details.

liberated_potato · February 14, 2021, 5:59pm

I have tried that in the past and couldn’t figure it out. Does that even work? Now I notice that they have added some description about that in the settings, but where and when do I get this document cards in my review ? Going through discord discussions, it seems like Document cards just show rest of the document along with the card I am reviewing, is that the case?

UMNiK · February 5, 2021, 10:15am

I am honestly just going off the setting’s description here, I’ve yet to see it popup in my queue (possibly because I’m not massive on separating my stuff into documents). But it certainly seems to be along the same lines you are thinking of, so might be worth investigating. Of course, it could also be bugged, but at least the devs are aware of the concept.

hannesfrank · January 26, 2021, 1:39am

I’m also interested in a screenshot of Document Cards for documental purposes, e.g in a (self answered) “What are Document Cards?” Q/A topic.

liberated_potato · November 23, 2020, 5:05pm

If you enable Document Cards today, you may see some of those cards the next day. I got one in my queue today (but the layout was messed up due to sticky headers) and I am quite convinced that they haven’t worked on it recently. In short, I think it is more like (conceptually) a ‘reveal all hide one’ type cloze, and the cloze here is a Rem. Here is a screenshot I got from discord

I like the intention here - to provide better contextual clue and sequential order (assuming it works), but I don’t know how effective it is to show rest of the rems in the document. Also, not sure if these are ‘additional’ cards on top of the normal cards, in that case this will increase the review load. I guess to use these, one has to make concepts into Documents, which doesn’t feel like the right way to organize notes. Above all, I don’t think it works reliably. So I think Document Cards is something that can be improved serve this purpose. Thanks for bringing this up @UMNiK

Parth · December 21, 2020, 8:18am

The feature I want to request is a way to study your Rem in chronological order rather than random order. This could help a lot when your trying to understand the information in your first attempt. I currently go through my Rem questions manually on a paper on the first attempt, then I study them in random order.

Thank You,

Perelyn · January 2, 2021, 9:48am

Use case: I use RemNote as a study tool in university. When I’m done summarizing a lecture I’ll test myself with the cards I created.

My problem: When I study a document, the flashcards appear in a random order. But the concepts in a lecture all build on top of each other. So when practicing concepts in the queue and I’m shown a flashcard concerning some concept that appeared in the middle or at the end of the lecture, then I won’t be able to answer it since I haven’t fully grasped and learned the concepts that came before it yet as they all build on top of each other.

Requested solution: Provide a possibility to practice flashcards as they are ordered in a document. This way I’ll be able to learn each concept one after the other, in the order they are intended to be learned.

liberated_potato · January 2, 2021, 9:48am

I totally agree with you, and that is exactly why we have this feature request Rem Cluster - a group of rems that can be reviewed together in sequential order. Please upvote if you think this will addresses the issue, or add comments if you have any suggestions.

For temporary purposes, you can disable all three clustering from the Queue settings and go to the document you are interested and study all the rems in it without spaced repetition. This is not perfect, but close enough. But to actually internalize stuff we need this implemented into the SRS system

Perelyn · January 2, 2021, 9:48am

Awesome, thanks for the link and the explation of the temporary solution! I really hope, they implement this feature soon!

hannesfrank · January 29, 2021, 4:10pm

As a workaround you might use the Active Recall Custom CSS (What is Custom CSS and how do I use it?).
thumb
You can try to remember stuff directly in your document and optionally after you finished practicing use the “Practice all cards in the Document” thing and just click Solid everywhere to record the practice session.

hugo · January 12, 2021, 2:37pm

Congrats for you thorough explanation. But I disagree with your desire of reviewing all the descriptor of a concept together. This idea goes against the principles o Spaced Repetition.

Going back to Piotr Wozniak (creator of SuperMemo in the 1980s) https://www.supermemo.com/pt-br/archives1990-2015/articles/20rules, you must stick to the minimum information principle. Each “slot” of information must have its own schedule. Otherwise, if you forgot 1 of 6 descriptors, will you have to drastically reduce the intervals of all of the 6 descriptors? That would not make sense.

Using Anki for a long time, I can assure you its even good to see these things separately. You will always be reactivating those related synapsis. And to have I full picture of the subjetc, RemNote has the great advantage that you can anytime open a new window and search for the original document (here is a drawback of RemNote - not having a button on the queue to directly opening the document in the rem that is being asked in the card, forcing us to open a new window and make a search).

But you can believe spaced repetition. It does not go against seing the big picture. Instead, it warns you when you should do this. When making a flashcard of one descriptor you fell you are not sure of the answer of another descriptor, you MUST solve this uncertainty immediately. Recalling randomically, with less context being supplied, is the key for enforcing your memory and truly mastering the content.

I prefer to think of RemNote as I very good tool to apply SR without losing the big picture. But messing the spaced repetition in not an option, at least as I can see.

liberated_potato · January 15, 2021, 6:43am

The recommended implementation (through Powerup) is not going to change any default behavior. It is going to be an additional feature that gives user the flexibility to decide at what level of granularity the randomization should happen. So it is essentially ‘if you don’t like, don’t use it’ feature.

Having said that, here are my thoughts on some of the points you mentioned.

The essence of minimum information principle is to create prompts

that are simple, precise and consistent
that you should be able to answer correctly almost all the time
that should light the same bulbs every time you review them

The easiest way to violate minimum information principle is by creating a prompt has more than one thing in it. However, in a rem cluster, we are reviewing only one prompt at a time. As long as the prompts inside a cluster do not violate minimum information principle there is no issue here. A cluster just means that the learner wants to think at a slightly higher level of abstraction. Only one issue here is the cost of forgetting a cluster. This cost is not same as cost of forgetting a complex item that Piotr talks about. It is because members in a cluster are not complex to begin with and members have their own schedule. I will explain in detail towards the end.

Also, let’s not confuse Spaced Repetition with flash cards. The benefit of SRS comes from spacing and repetition. You can apply SRS at any level. One popular example is Ali Abdaal’s Retrospective Revision Timetable in which he applied SRS at chapter level. Flashcards implement SRS. e.g. Anki is a software that implements SRS at flashcard level, but RemNote is way more than just a flashcard software. In its true essence SRS only deals with scheduling (i.e. intervals) of the units of knowledge it is dealing with. The size of the fundamental unit is up to the learner. It can be a flashcard or a small cluster or something even bigger. I even say that SRS applied at flashcard level (and the typical Anki style flashcard reviews) is prevalent because initial tools like Anki and SuperMemo popularized that idea. Even RemNote borrowed that idea and innovated it by adding hierarchical context to the cards (which is great). But ultimately it is similar to how people kept innovating Internal Combustion engines for centuries. RemNote can do more than just imitating Anki. With outliner capability, the new graph feature and all other great features, RemNote can change the way we learn.

I have suggested a solution for this at the bottom of the request, under What if I forget one card from a cluster?. The funny thing is, the idea of a cluster already exists in some sense, even in Anki, it is called a List. A list schedules a small group prompts together and in sequential order. So the worst case penalty is same as forgetting a list. But we can come up with better algorithms to deal with this. One such case I mentioned is, if I forget a member from the cluster, that member is considered forgotten and gets kicked out (temporarily) of the cluster. Now to schedule reviews for this forgotten card, we can take into account the next repetition interval of the cluster and plan the relearning phase of this forgotten card in such a way that one of the forgotten card’s reviews aligns with the next review of the cluster. So when the user reviews the cluster next time, the forgotten card will also become part of the cluster review. If the forgotten card gets answered correctly, then from that point onwards we can schedule the forgotten card along with the cluster. So the cluster idea gives the user a chance to test if the forgotten card can actually get back to its original schedule, thereby actually reducing the number of future reviews of the forgotten card. This idea can be applied even to lists to actually reduce the penalty of forgetting lists, especially when we forget just one or two members of the list. This is just rough idea, there can be corner cases to this and this will make scheduling complicated from the tool implementation standpoint.

While I agree with the fact that random recall will put more intellectual load on the memory and potentially result is better enforcement, only downside is the practical aspect of it. For example, if I am seeing a card after 6 months, it will be hard to just sit there and think about all its sibling cards. All I want to do at that point is to answer the card, complete the review and get on with life.

The bottom line is, everyone learns differently. Above all, this is just an optional feature.

88DM88 · January 17, 2021, 10:02pm

Am I just unlucky or are clusters already a thing? For example I get the descriptors of a parent in groups and all the clozes as well. I even turned off all cluster settings yet I still experience this issue

liberated_potato · January 19, 2021, 1:42am

As you probably know, there has always been some level of ordering by default. I think the ordering depends on how many cards are there in the queue, what are the card types, in what learning phase those cards are in and also what is the state of Cluster settings (these are just my observations). If you have just descriptors and those cards are new, there is a good chance that you will get them in sequential order

Actually, we also need some improvements to prevent sequential ordering where it doesn’t make sense. For example this feature - Bury related new card until next day from last June (from Discord) and Bury (automatically) other cloze cards on the same rem

Subham_Mohanty · January 19, 2021, 9:05am

I use this workaround using List cards for the time being

blablagang · January 29, 2021, 4:45pm

Accidentally found this amazing feature! Thank you!

blablagang · January 29, 2021, 4:47pm

Do you think you can connects this to SM2 Algorithm? Or maybe simple ones like adding a reminder of one day, one week or one month automatically after review?

Or maybe a button. After reviewing the note, showing a button so there won’t be any need to ##learned.