If you focus on the generative part, GANs and VAEs are actually mathematically the same object (1), i.e. Gaussian latent variable models, where $z$ is a latent Gaussian random variable pointing to an observed $x$:

The difference is that VAEs are prescribed models that output a random variable $x$ with a probability density, while GANs are likelihood-free implicit models (2) that directly specifies a (deterministic) procedure with which to generate data.

Concretely, the VAE's graphical model is implemented as the decoder/inference network, while the GAN's graphical model is implemented as the generator network; the GAN's discriminator network does not appear in the graphical model (similarly to how the VAE's encoder/recognition network doesn't show up) because it is merely an auxiliary object created to approximate the Jensen-Shannon divergence or other f-divergences (1):

(Image from Lilian Weng)

References

(1): I've linked the relevant timestamp of the video recording of a tutorial (slides here) by Shakir Mohamed and Danilo Rezende from DeepMind at UAI 2017; Ferenc Huszar also explains the equivalence on Reddit. The VAE's graphical model is also explained in Stanford's "CS236 Deep Generative Models" notes.

(2): The distinction between prescribed and implicit models is described in greater detail in "Learning in Implicit Generative Models" by Shakir et al. (2016).

Extra: Another perspective with augmented graphical models

On Unifying Deep Generative Models (Hu et al., 2017) illustrates augmented graphical models that encompass both the GAN generator and discriminator, and an analogous model for the VAE where we assume a perfect discriminator.

Arrows with solid lines denote generative process; arrows with dashed lines denote inference; hollow arrows denote deterministic transformation leading to implicit distributions; and blue arrows denote adversarial objectives.