Reversible tyrosine phosphorylation is an essential process in many signal transduction pathways, and it is the balance between the actions of the kinases and phosphatases in the cell that are responsible for the phosphorylation status of these pathways. A key question in the field is how phosphatases achieve specificity for their substrates. Protein tyrosine phosphatase 1B (PTP1B) is a non-receptor tyrosine phosphatase that is known to negatively regulate insulin and leptin signalling by dephosphorylating the insulin receptor kinase (IRK) downstream of insulin, and Janus Kinase 2 (JAK2) downstream of leptin. While PTP1B has been studied in great detail in the case of its interactions with IRK, no studies have detailed the interaction between PTP1B and the JAK proteins. In this study we investigated how PTP1B interacts with JAK2 using structural and biochemical studies. Crystal structures of the PTP1B phosphatase domain in complex with the JAK2 activation loop suggested that the second phosphotyrosine of the activation loop is the preferred tyrosine for the catalytic site. Biochemical data further supported the crystal structure, revealing that the second phosphotyrosine is a better substrate. Our crystal structure of PTP1B in complex with the JAK2 activation loop peptide differs from what is seen in previous structural studies of PTP1B in complex with an IRK activation loop peptide where the first phosphotyrosine sits in the catalytic pocket and the second in the aryl binding site, which has been well described for PTP1B. Our findings suggest that PTP1B has mechanisms of substrate recognition which have not previously been described for this enzyme. These data provide useful information regarding the mechanism of ligand recognition and dephosphorylation by PTP1B and further our understanding on regulation of kinase substrates by PTP1B.