In this paper we extend the theory of conformal graph directed Markov systems (which by definition need to have finitely many vertices and are allowed to have infinitely many edges). Our extension, referred to as (conformal) pseudo-Markov systems, allows infinitely many edges as well as infinitely many vertices, and therefore goes clearly far beyond the class of graph directed Markov systems. For these systems we then develop appropriate symbolic dynamics, prove a version of Bowen's formula for the Hausdorff dimension of their limit sets, obtain a formula for the closure of their limit sets, and show that there always exist conformal measures along with their invariant versions. Subsequently, we use the obtained analysis of conformal pseudo-Markov systems to show that there is an interesting, rather exotic class of infinitely generated Schottky groups of the second kind (acting on (d+1)-dimensional hyperbolic space). This class contains groups having limit sets of Hausdorff dimension equal to any prescribed number t in (0,d], whereas the Poincare exponent can be less than any given positive number s strictly less than t. Also, we show that the dissipative part of the limit sets of these groups has some interesting properties.