\section{Non-ergodicity between non-absorbing states}
\begin{definition}[Non-ergodicity between non-absorbing states]
Assume that $N$  exists for any policy $\pi$
 and is independent of initial states.
 $\exists i,j \in S\setminus\{\text{T}\}$, 
 $N_{ij}=0$, MDP is non-ergodic between non-absorbing states.
  \label{definition3}
\end{definition}




\subsection{St. Petersburg paradox}



The St. Petersburg paradox is a paradox associated 
with gambling and decision theory. It is named after the city 
of St. Petersburg in Russia and was initially introduced
 by the mathematician Daniel Bernoulli in 1738.

The paradox involves a gambling game with the following rules:
\begin{itemize}
  \item Participants must pay a fixed entry fee to join the game.
  \item The game continues until a coin lands heads up. 
Each toss determines the prize, with the first heads
 appearing on the $t$-th toss resulting in a prize of $2^t$.
\end{itemize}


%\input{pic/FigureParadox}

The expected return of all possibilities is
\begin{equation}
\begin{split}
\mathbb{E}(x)&=\lim_{n\rightarrow \infty}\sum_{t=1}^n p(x)\times V(x)\\
&=\lim_{n\rightarrow \infty}\sum_{t=1}^n\frac{1}{2^t} 2^t\\
&=\infty 
\end{split}
\end{equation}


Despite the potential for the prize to escalate 
significantly, the expected value calculation 
in probability theory reveals that the average 
participant in this gambling game would end up paying
 an infinite fee. This is due to the prize's expected 
 value being infinite. Even though the probability of
  winning is small with each toss, when multiplied,
   it leads to an infinitely increasing expected value.

This paradox challenges individuals' intuitions and 
decision-making regarding gambling. Despite the allure 
of a potentially substantial prize, the actual expected
 value of participating in this gambling game is infinite.
  Consequently, in the long run, participants could face
   an infinite monetary loss.

\input{pic/paradox}

Figure \ref{TruncatedPetersburg} is a truncated version
of the St. Petersburg paradox. The transition probabilities between
non-absorbing states are as follows:
\[
Q_{\text{st}}\dot{=}\begin{array}{c|ccccc}
 & \text{S}_1 & \text{S}_2 & \text{S}_3 & \text{S}_4 & \text{S}_5  \\\hline
\text{S}_1  & 0 & \frac{1}{2} & 0 & 0 & 0 \\
\text{S}_2 & 0 & 0 & \frac{1}{2} & 0 & 0 \\
\text{S}_3 & 0 & 0 & 0 & \frac{1}{2} & 0 \\
\text{S}_4  & 0 & 0 & 0 & 0 & \frac{1}{2}  \\
\text{S}_5 & 0 & 0 & 0 & 0 & 0   
\end{array}
\]
Then,
\[
N_{\text{st}}=(I_5-Q_{\text{st}})^{-1}=\begin{array}{c|ccccc}
& \text{S}_1 & \text{S}_2 & \text{S}_3 & \text{S}_4 & \text{S}_5  \\\hline
\text{S}_1 & 1 & \frac{1}{2} & \frac{1}{4} & \frac{1}{8} & \frac{1}{16} \\
\text{S}_2 & 0 & 1 & \frac{1}{2} & \frac{1}{4} & \frac{1}{8} \\
\text{S}_3 & 0 & 0 & 1 & \frac{1}{2} & \frac{1}{4} \\
\text{S}_4 & 0 & 0 & 0 & 1 & \frac{1}{2} \\
\text{S}_5  & 0 & 0 & 0 & 0 & 1 \\  
\end{array}
\]
Bases on Definition \ref{definition3}, 
the truncated  St. Petersburg paradox
is non-ergodic between non-absorbing states.

\subsection{A sufficient condition for non-ergodicity between non-absorbing states}
By observing the truncated  St. Petersburg paradox, 
it is easy to provide a sufficient condition for non-ergodicity between non-absorbing states.
\begin{theorem}[A sufficient condition for non-ergodicity between non-absorbing states]
\label{judgmentTheorem}
Given a Markov chain with absorbing states, 
suppose the size of the non-absorbing states $|S\setminus\{\text{T}\}|\geq 2$.
If the transition matrix $Q$ between non-absorbing states satifies,
\begin{equation}
\forall i,j \in S\setminus\{\text{T}\}, Q_{i,j}=\begin{cases}
\geq 0, & \text{if } i\leq j; \\
 0, & \text{otherwise.}  
\end{cases}
\label{condition}
\end{equation}
Then, the Markov chain is non-ergodic between non-absorbing states.
\end{theorem}
\begin{IEEEproof}
The $Q$ matrix (\ref{condition}) is an upper triangular matrix.
The product of two upper triangular matrices is still an upper triangular matrix.
Furthermore, the sum of two upper triangular matrices 
is still an upper triangular matrix.
Based on Definition \ref{definitionN}, 
the $N$ matrix is product and sum of upper triangular matrices.
Then, the $N$ matrix is an upper triangular matrix.
The claim now follows based on Definition \ref{definition3}.
\end{IEEEproof}