There are two organization mechanisms within rules that you can use to help keep your rules nice and tidy. Every rule is capable of functioning entirely on its own, however there are many instances where you may have two or more rules that work together to complete a task. In these cases, it's a good management practice to keep these rules organized together so that they can be easily updated, imported, exported, enabled, disabled, and/or debugged.
Rules in a rule group always share the same event. When rules that are triggered by different events need to be organized together, a rule "set" should be used. Any rule can easily be turned into a rule group by clicking the button with the "plus" icon (which will add a "sub-rule").
When you add a new "sub-rule" to an existing rule, the rules together become a "rule group". The sub-rule(s) of the parent rule will only be evaluated if the parent rule conditions are met. What this does is allow the sub-rules to essentially inherit any conditions put on the parent rule since if the parent rule conditions are not met, then the sub-rules will not be processed either. Of course, additional conditions may also be placed on the sub-rule(s) to further limit the situations for which actions attached to the sub-rule should be taken.
Any time that two rules share the same event and one or more conditions, then that is an opportunity to organize them into a rule group. When you are creating a rule group, it is important that the parent rule only contain the conditions which are common to all of the sub-rules. The benefit of organizing rules into rule groups is that when you need to tweak a condition which is common to the whole group, you only need to do it in one place (the parent rule) instead of having to keep the same condition updated on each of the sub-rules.
Rule sets allow you to organize rules which work together as part of the same purpose or goal. A rule operates the exact same way inside of a set as it does outside of a set, however, placing rules into a rule set allows you to easily enable, disable, export, delete, or debug all of the rules in the set at the same time. In other words, rule sets are like "folders" for your rules to be filed in.
As an example, you may have a series of rules set up on your site that are designed to help keep members engaged by creating "congratulations" notifications when they reach certain milestones, and then sending follow up emails if they become idle. Those rules could be placed together inside of a rule set titled "Member Engagement Rules" for convenience.