Let $(X,\S)$ be a set system on an $n$-point set $X$. The \emph{discrepancy} of $\S$ is defined as the minimum of the largest deviation from an even split, over all subsets of $S \in \S$ and two-colorings $\chi$ on $X$. We consider the scenario where, for any subset $X' \subseteq X$ of size $m \le n$ and for any parameter $1 \le k \le m$, the number of restrictions of the sets of $\S$ to $X'$ of size at most $k$ is only $O(m^{d_1} k^{d-d_1})$, for fixed integers $d > 0$ and $1 \le d_1 \le d$ (this generalizes the standard notion of \emph{bounded primal shatter dimension} when $d_1 = d$). In this case we show that there exists a coloring $\chi$ with discrepancy bound $O^{*}(|S|^{1/2 - d_1/(2d)} n^{(d_1 - 1)/(2d)})$, for each $S \in \S$, where $O^{*}(\cdot)$ hides a polylogarithmic factor in $n$. This bound is tight up to a polylogarithmic factor and the corresponding coloring $\chi$ can be computed in expected polynomial time using the very recent machinery of Lovett and Meka for constructive discrepancy minimization. Our bound improves and generalizes the bounds obtained from the machinery of Har-Peled and Sharir (and the follow-up work by Sharir and Zaban) for points and halfspaces in $d$-space for $d \ge 3$.