Reinforcement learning (RL) methods have received much attention due to impressive results in many robotic applications. While RL promises learning-based control of near-optimal behaviors in theory, successful learning can elude practitioners due to various implementation challenges. Even if the best-suited learning method was selected, learning performance can nonetheless disappoint due to badly chosen hyper-parameters or an unreliable implementation of the algorithm. Furthermore, a learning task can be made unnecessarily hard by incorrect specifications. This full-day tutorial points-out these practical pitfalls and introduces the audience to the tools for robotic RL that will aid roboticists in successfully solving robotic learning tasks, both in simulation and the real-world.