Toward the beginning of the book of James, it tells us that we need to be doers of the word and not merely hearers who delude themselves (1:22f). It also reminds us that our faith is to go beyond mere mental activity (2:14f). Works are what is supposed to demonstrate true faith. A major part of doing the word involves keeping what James refers to as the "Royal Law," which is to "Love your neighbor as yourself" (2:8f). If you do not keep this command, you become a lawbreaker and are therefore guilty. Even if you have kept all the commands, and yet break just this one command, you are still guilty as a lawbreaker (2:10). But, if we keep the Royal Law, James says we do well. Why? Because so many of the commandments fall under the umbrella of the Royal Law, "Love your neighbor as yourself."
One group of commandments that falls under this umbrella has to do with speech. The last set of instructions concerning speech comes from 4:11-12. It tells us not to speak against of judge a brother. If we speak against a brother or judge a brother, we are no longer a doer of the law. Why? The reason is because we have broken the Royal Law in not demonstrating love for our brethren. James also has reminded us that our religion is worthless when we do not bridle our tongue (1:26). We defile ourselves by the words that come out of our mouth (3:6). Things such as fighting, backbiting, conflict, etc., come from the worldly cravings that wage war within us (4:1). They are not displays of God's wisdom (3:13-18), but of worldly wisdom and friendship with the world is hostility towards God (4:4).
This should be enough explanation and motivation to refrain from speaking against or judging a brother. But James adds this. When you speak against or judge a brother, you become a judge of the law and not a doer of it (4:11). In other words, when you speak against or judge other servants of God, you set aside the Royal Law, thereby judging it as not applicable to yourself. You become a lawmaker by adding your own exception clause to it. Becoming a judge and lawmaker in this way puts you at odds with the true lawmaker and judge. Only the true lawmaker and judge has a right to set aside the law. However, when he does set aside the law, he does so in the interest of grace and mercy, not judgment and condemnation.
Living under the law of love, we are to "Let no evil talk come out of your mouths, but only what is useful for building up, as there is need, so that your words may give grace to those who hear" (Eph 4:29).
This is part of what is involved in being "doers" of the word. Keeping the Royal Law is not just about merely speaking the law, but putting it into practice. Based on all of this, you could add this summary that boils down a major part of the teaching in James: "Be not judges of the law by setting aside any part of the law, but doers of the law by fulfilling the Royal Law, which is to love your neighbor as yourself."