I think one of the great ways to explore the city is to get on one of these trolleybuses in Yerevan and make a full trip. These buses come in a few colors and while some of them are very old (like the red ones), others are renovated and more comfortable to travel on (like the green ones). Most of them are from the Soviet era, so you also get to feel the Soviet atmosphere. Besides riding in a means of public transport like this, it brings you closer to the locals.
Importantly, these buses run on electricity; they are cheap and quite slow – slow enough for you to see the city while riding. And of course since this is public transport, they will take you to residential areas, away from the touristic spots. This is something to consider if you have already been to all of the touristic spots and are open for new adventures. Just make sure that you take a full ride so that you come back where you started from (and to avoid getting lost in the residential areas). Such a full trip may take an hour or so. Usually a good location to take the trolleybus is Mashtots avenue in the center of the city.