[Voiceover] National Data Archive on Child Abuse and Neglect [Erin McCauley]  And so welcome to the Summer of NYTD everyone. This is a summer training series that we're doing focused on one specific dataset, The National Youth in Transition Database, and we are really glad everyone can be here. Just again please turn your video off and mute your line, and FYI the session is being recorded so we can make it into a video. So the NYTD dataset is in The National Data Archive on Child Abuse and Neglect in the Bronfenbrenner Center for Translational Research at Cornell University. All of our sessions will focus primarily on the NYTD dataset. It's a great dataset, you guys are about to learn a lot about, I promise. And so here's an introduction just for the summer training series, my name is Erin McCAuley and I'm going to be  the host of the series. so you'll hear me at the end and the beginning of every session and I'll also be leading two as well. And so our sessions for the summer will be in August and September. They'll be on Wednesdays always from 12 to 1 PM at the latest. Today's presentation is an introduction next week I'll be talking about the data structure, so I'm talking about the service files versus the outcomes files. I'll also be looking about the different resources we have at the BCTR to help people use the dataset. Our next two presentations will be led by Michael Dineen. He's one of our house experts. His first session will be preempting the typical questions that we get from data users and then his third one will be talking about linking and so that's a really special capacity of our dataset as we can link it to two other datasets that we house in the Archive. And then to round out our series we're going to have two research presentations. I'll be giving the first one and it will be an in-progress research study so it's something that I'm working on right now and it will be kind of messy but working through it I'll share with you guys what I found so far from using the dataset. And then our very last one will be a very polished published research study. So now I'm going to pass off the presentation to our wonderful presenters Telisa and Tammy. We're really lucky to have them here today to give us an introduction to the dataset and to kick us off. [Telisa Burt] hello my name is Telisa Burt I'm a social science analyst with the Department of Health and Human Services and I am currently serving as the project manager for NYTD, the entire portfolio, all the way from data systems development through to the data reporting compliance and review. And I have with me Tammy White if you'd like to introduce yourself. [Tammy White] hi everyone! This is Tammy White I'm with the office of data analysis research and evaluation and I am really responsible for analyzing all the NYTD data and manage the data itself and do some analysis on it. [Telisa Burt] okay so the NYTD, The National Youth in Transition Database, collects case level data for young people who've had experiences in child welfare and it helps us understand their experiences and how well they are prepared for adulthood. NYTD, states began collecting NYDT data in 2010 and submitted for the first time in 2011 and so NYTD data collection was authorized along with the John H. Chafee Foster Care Independence Program and that was all that and much of our child welfare programs were guided by the Social Security Act, however in 2018 The Family First and Prevention Services Act was signed into law and that served as a reform and guidance for all of the programs that you may have heard of before. So Title IV E and Title IV B as well as formally Chafee Foster Care Independence program. So what information is collected? So there are 58 data elements collected in NYTD and those elements capture demographic data as well as service subscriptions and we try to see how young people you know we try to learn about who the young people are that come into our care and that leave our care. And we try to see what types of services were available to them and what types of services they actually participated in. And then we have six outcomes that we capture through a survey and that survey captures that data at age 17, 19, and again at 21. And for those outcomes we're looking for financial self-sufficiency, educational attainment, homelessness, risk factors, connections with adults, and access to health insurance. Okay so again NYTD captures experiences of youth who are involved in child welfare and it helps us helps provide us with a national picture of those youth who are aging out of care and how they fare. So much of this data can be used to follow young people through various child welfare programs since we use a common identifier so we can track them through NCANDS, AFCARS, and through NYTD to kind of see how this shared population fares and the services that have been delivered to them. So another way that we use NYTD data are for program improvement. It allows us to think about whether there are gaps in services and whether we have a high or low service subscription and it allows us to take a better look at how effective and efficient our services are. Catherine Heath is our resident expert and program manager for the Chafee program and so she relies on a lot of this data to tell us how well our state programs are operating. Now one of the most important aspects of NYTD data are how involved and engaged our youth are First of all the survey allows young people to self-report their conditions and I think that's a powerful opportunity for us to hear directly from young people themselves about their experiences and where they are in life. And it allows us to think about how we engage those young people differently in the future. And so one of the best parts about this process is that almost every step of the way we engage youth in program delivery we engage young people around the reviews of our state partners, and we take a lot of opportunity to hear from our young people not just through the survey that we administer through NYTD but in the development and administration of those surveys as well. So that is a very important key to most of our NYTD work. And as you all are aware, once you're capturing data there are opportunities for a longitudinal assessment and we can use the data to influence policy and program design and we have made it accessible through Cornell for researchers to be able to use it and start some innovation in the field which were hopeful and very grateful that this series helps to introduce this data to you all for those purposes so it's wonderful that you all will have an opportunity to look at our data and see how you can use it to improve the work in the field. So we are guided through regulation on compliance and we collect data on a semiannual basis so at those two points in time we look to make sure that the data are accurate, that it is timely and structurally sound. And once data compliance is assessed, states do have an opportunity to resubmit a corrected file And so we look at things like data quality in terms of error rates, we look at participation rates There are a number of elements that we review and assess to determine compliance, and once and if a corrected file is submitted and our state partners meet compliance standards then we understand that we have quality data however in the event that they are not able to make improvements to their data then they are assessed a penalty. And then once they are assessed a penalty we do employ technical assistance to help them work through their issues and and sometimes it's a system oriented problem or sometimes we have to help them again learn how to engage young people differently so that they can improve participation rates. Or you may work with them on how they are actually capturing and inputting data in the event that it is an error for training. One of the other areas to support data quality are on-site reviews and between 2014 and 2016 we conducted a total of 12 reviews since 2014. And the beginning of those reviews were pilots so we had a few states who agreed to participate on reviews and the information that we were able to gather kind of inform the process moving forward. And so now what we do is we take the federal team on-site to our various states and it's composed of central office staff, regional staff, technical staff and as I explained to you earlier about our young adults and engaging young people we do have young people who are formally a part of child welfare systems and they have been well trained on how to review as well and so they join us in partnership as we go on site and review states. And so the goal is to see if their policies and practices relating to data collection are appropriate and we validate and verify their information systems to make sure that it can collect the information that are required. And then we assess their survey methodology and again this is where engagement is important and participation rates are impacted based on states' ability to reach out to young people and get them to follow up not just when they're in care at 17, but then again at 19, and again at 21 and most of you with research backgrounds know that that is typically the response rate is usually a challenge when you start to get into longitudinal studies. And then while we are on-site we actually explore how the states use their data because continuous quality improvement is important to us. We don't want them to just capture data because it's a requirement. We want to see how it is integrated into their program delivery moving forward. So those are some of the important aspects of the data that we collect with NYTD. Additionally in terms of data quality, again as I said not only do we have the compliance piece we also have the technical assistance and help desk which helps train and support our state staff and state partners who are trying to learn how to input data correctly and how to use their NYTD system correctly. But we also have a program, the NDRU (NYTD Data Review Utility), which is a technical program that allows our states to run their data through the compliance process to make sure that it meets our quality data standards before they submit and so there are many steps to the process of ensuring that we are producing data quality. And so I'll pass it on over to Tammy as we talk about how and when data are collected and what that data looks like. [Tammy White] So hi everyone this is Tammy White as I said earlier I'm primarily responsible for managing and analyzing the NYTD data sets themselves and I know that probably the next session when Michael Dineen goes over the data sets you'll have a little bit more detail but we did want to try to find an opportunity to give you a broad overview but please feel free to ask specific questions in the chat box. As Telisa mentioned, NYTD has been around since fiscal year 2011 we have surveyed if you can tell we had three cohorts so far. Cohort one is complete we had the 17-year-olds surveyed in 2011 and then 19 at 2013 and then 21-year-olds in 2015. I don't need to go through the chart for you but you can see that we are just about finishing up to 21-year-olds for cohort 2. We should be the end of the survey wave for that one is September 30, 2018. And then we just completed the 17-year-olds of cohort three so we've got some good trend data opportunities to look at to see if 17-year-olds are comparing as we go forward in cohorts and we can do some good longitudinal analysis. That first line is the services information. Again, next time you'll hear a little bit more about how the data sets are structured through NDACAN but states report to us every six months the number of services they provide to youth, and that is actually reported to us every year so we don't have any gaps in the services information and that is available from 2011 to 2017 so far and then you know once the 2018 data is done that will be available to the archive as well. Much of this presentation I didn't spend a whole lot of time on services information but as Telisa mentioned the services are reported to us every six months and there's, there's about 14 categories of services and you saw that on one of the earlier slides. There really isn't a whole lot to say about that other than results and we can tell you a little bit about that but this is really a more broad overview. So the first one is the baseline population. So the baseline population is states report to us all youth who have reached age 17 while in foster care. And so they report that population to us so that we can what the eligible population would be of the youth for 17 who were eligible to be surveyed. And then that from those, those youth who participated age 17, they become like the cohort that we follow forward at ages 19 and 21 and they are a subset of that bigger population that has been eligible because not every youth obviously is able to participate. And so of those 17-year-olds who participated we have a group of the cohort who were in that follow-up population. And so to be eligible to follow up at age 19 you had to have participated in the survey and the state would have had you participate within 45 days of turning age 17, and you had to have been in foster care at the time of taking that survey. So that sort of narrows the population down a little bit and the intention of that was to really try to gauge the experiences of youth at baseline while they're in care and as close to turning age 17 as possible so that we can get a sense of where they are before they leave care. Many states now I think were up to 26 states I believe who have extended care until age 21 but at the time that NYTD was around we really wanted to make sure we had that foster care experience because many states were aging out at 18. And in addition the survey has to be a valid survey which means at least one survey question had been answered. The youth cannot decline every one or have all blanks and then the not applicable options were answered so at least one survey question had to be answered in order to be counted. And then at age 21 and it's that same group of 19-year-olds but obviously not including those who have been reported to be deceased at age 19. There's a subset of it gets complicated a subset of this follow-up population states have the option to follow a sample of 19-year-olds and that was really intended to reduce some of the burden on the states who have very large populations of youth in foster care at age 17. And so to follow thousands of young people forward we there was before my time but there was an advisory group and people got together and said you know let's make it so that the states that have large populations if they choose to follow a smaller group of people forward at ages 19 and 21 they have the opportunity to do that. And so at ages 19 that for the states that opt to sample those are the youth that are in that follow-up population and those are the same youth that same sample of youth who are also in the follow-up population at age 21. And I'll talk a little more about the sampling and so as I said the frame for that is all the eligible youth who are eligible to be followed up at age 19 which is all those 17-year-olds who took the survey and the calculation which is in the regulation in appendix B and it's also detailed on the Children's Bureau's website technical bulletin number five we take a random sample of those youth and then add 30% to that number so to allow for some attrition at age 19 so that we don't start out with you know if your sample size is really 100 then you lose a bunch than we want to make sure we build in a little bit of a cushion for that. So the sample sizes are set at that confidence level plus the 30%. And so any state can sample as long as their base population their cohort of 17-year-olds is large enough. Roughly that means at least 85 youth. Most of the states have well over that they choose not to follow up anywhere from 100 to almost close to 1000 youth depending on the size of their caseload. In cohort one we had 12 states that sampled and you can see what they are there. Cohort two we have three more states added to the mix so it's those original 12+ Colorado, Maryland, and Missouri. And so cohort three we don't have all the states in yet to have chosen to sample so far I think we've only got like eight or nine requests but they have a little bit of time left so we'll see who ends up doing it for cohort three. And just quickly about the NYTD datasets again and I'm sure next time Michael will get into much more detail but in general we've got this services files that go from fiscal year 2011 through 2017 available at the archive. Roughly 100,000 youth each year are reported to have received at least one independent living service so I mean that's fairly consistent trend wise over those five years six years that we have seen states reporting so they're doing a really nice job of reporting the number of youth who are receiving independent living services. And so for the outcomes information as I mentioned cohort one is now complete they were at 17, 19 and 21 and that ended in 2015. In cohort one we had roughly 16,000 youth participated age 17 which turns out to be about a 53% participation rate which in the first time out of the gate for states people think you know it can be a little low but overall I think states were trying very hard to be prepared for it and some states did better than others but we had a 53% response rate And then once out of that 15597 those 12 states that sampled that have allowed for roughly 11,700 youth to be eligible to be followed up at 19 and of those there are roughly 1700 who participated which gave us about a 67% response rate. And again the eligible population at age 21 dropped by 37 and those were 37 youth who were reported to have been deceased at age 19 so that denominator is smaller and roughly 6900 of them participated with just about a 60% response rate. And in cohort two as we kind of expect as states were finalizing an sort of feeling out their methodology and getting their processes together cohort two is a bit better a little over 16,000 completed the survey out of those roughly 23-24,000 which is a 69% response rate which is quite a bit better than the 53 we saw at the 17-year-olds in cohort one. And then in cohort two almost 9000 out of the 12,000 and that's up to a 72% response rate for age 19. So cohort two is quite a bit better than cohort one. I think it's a reflection of how well states are doing and how seriously they're taking the survey processes and trying to get keep track of young people and trying to keep them engaged in and trying to follow up with them especially it's very difficult to follow up with youth at age 19 although like I said some states are doing a pretty good job of keeping their youth in care at age 19 so their response rates I think are being reflected in that. And so actually before we get to the questions I did I did want to mention one thing that I neglected to put a slide in on here and I'm again I'm sure Michael will go into more detail in the next session when it comes to analyzing these the datasets we have done some weighting so when you look at those response rates that I mentioned we've tried to account for some nonresponse bias. What we use is the ability we have the ability to match up the NYTD data with the AFCARS data and so we've used some of the AFCARS information on these youth and put it into a weighting methodology and try to correct for some of that nonresponse bias. What we did find is that the results aren't drastically different from the unweighted results and so the nonresponse bias in terms of the variables that we captured the variance isn't really being captured at the child level so it doesn't appear in necessarily the characteristics of the child that is causing some of these response rates and it's probably more at the state implementation and we don't collect data on that. That isn't part of the weighting methodology and there's a weighting document on NDACAN's website and again I'm sure Michael and NDACAN will go into much more detail about that but I did want to mention that in terms of the response rates that we do have we do have some weights in those datasets to try to account for some of that. And I see that we breezed through that presentation in like a half-hour but we really did want to give people some time to ask questions and have a bit more of a dialogue about this rather than inundate you with details at this stage. [Erin McCauley] great so thank you guys so much for that presentation it was very informative. So if anyone has any questions at this point either you can unmute yourself and ask the question and kind of engaging dialogue in that manner or if you don't have audio setup or you don't want your voice to be recorded you're welcome to send in a question on the group chat and then one of our presenters will read it out loud and answer the question. [Tammy White] I just want to point out in the meantime that the slide that you're looking at is actually really helpful for resources that's if you go to the CB website the Q and a questions especially are very helpful if you want to know the nuances of who gets reported and who doesn't and why certain youth are in or out. And then we do have some data briefs up there where in the process of doing a couple more which hopefully will be up soon. And then the federal guidance and the information on our NYTD reviews as well is out there. We also have a couple tables, state-level tables that are available and that doesn't look like that made the slide but if you go on the CB website and you look under the reporting system tab under NYTD there's a link there that will take you to some outcomes and services reports that are state-level tables. And those are 2016's I believe and I am in the process of updating that with the 2017 information. [Erin McCauley] so we have a few questions that have come in. In regards to when the video will be available and access to the slides the video will be available probably in a few weeks but we are going to have the audio transcribed so that folks with hearing impairments can still follow along. And so both the video and the slides themselves will be posted on the national data archive on child abuse and neglect website www.ndacan.acf.hhs.gov and so we can send out a blast to the people on our list and if you haven't signed up for our email listserv I suggest you do because that's where most of our communication will come through. I know that our flyer got sent out to a couple different listserv's but our NDACAN listserv that's where will have the main information and we can send out a blast once they get posted online. And there is a few other questions. [Telisa Burt] okay so do you all pull data from OPPS for NYTD data? [Tammy White] I don't know what OPPS stands for does anybody? Oh Opportunity Passport Participant Survey, thank you. Yeah no we don't so we receive the files from the states themselves and so how the states conduct or provide the survey to their youth is dependent on the state. We don't dictate the methodology and so whether part of that is folded in some states may fold it into that process and use the NYTD questions along with the OPPS and then extract those NYTD questions to us that may very well be possible but we don't pull it directly from there it's just states report only the NYTD information to us and that does go along I think with the NYTD plus instrument question again some states use it some states don't. Michael may have a better idea on how many states actually use it. We don't see that information so states will use that and then when they send us the file they extract the relevant NYTD elements to us and then leave the plus  to themselves. [Telisa Burt] okay there was another question about the number of youth who passed: have you found a relationship to their deaths in foster care? [Tammy White] so in answer to the death in foster care question I have not looked at that and so it's 37 youth and honestly I haven't delved into those 37 to see where they were at age 17 in terms of their foster care experiences. But it certainly can be done since it can be linked with AFCARS I don't know what happens between 17 and 19 obviously but it certainly you could get a sense of where they were at age 17 on that but I have not specifically done that analysis. [Telisa Burt] okay our states doing all of the collection and reporting methodology here, are these processes adequately harmonized or comparable from state to state? The collection process varies from state to state and we definitely invite in and provide access to resources that help states learn more about best practices in that arena and many states may have their state workers actually collect the data and train them to collect the data and in other instances they contract that work out to local universities or other organizations that are bit more well-versed in data collection and methodology. [Erin McCauley] for signing up for the emails or the listserv you can do that through the website which is www.ndacan.acf.hhs.gov. Also if you just Google the national data archive on child abuse and neglect it comes up. And through there is where you can access all of our resources including signing up for our listserv. If you get to the website it's one of the top across the top is the listserv and then how to sign up. [Telisa Burt] And Tammy I think this one is for you since you do the national analysis for us but the question is would you advise against creating national totals or should the data be kept at the state level due to differences in methods? [Tammy White] you know we use national totals I mean I think it's the best data we have and given we've tried to account for some of the differences in the weighting and then again at that next weighting session Michael will go into more detail about you know even constructing your own weights with different information that maybe we don't have. But no, I think national totals are fine and although the numbers have increased in say cohort two the trends are mostly the same so I think whatever states are doing they're doing consistently and so I think the totals are probably accurate and consistent as best as we can do it. I would you know love if people out there were able to pull in a variable of methodology and do some analysis on that. I don't have access to that but I do think that that's an important variable to account for if it could be captured but some states do mix methodologies I mean for follow-ups they do phone calls and send for kids who are still in care they will do them in person and then some do them through you know websites in addition to interviews so some of it is difficult to capture too because within states the methodology varies as well. [Erin McAuley] and so I shared my whole screen now so you guys can see the website. The listserv sign-up is here and then under publications we have a system where you can search our publications and see how other people have used the data. If you're interested in using data I suggest it there's some really great work that's been done with it and people are making national estimates so here's the website. [Telisa Burt] is there a way to track you with disabilities and their outcomes? [Erin McAuley] yes there are. So as part of the data collection we mark if they have a disability. And actually our first research presentation that I'm doing will be related to youth with disabilities who are in the NYTD data set. [Michael Dineen] you can also link the NYTD data to AFCARS foster care file where there's information about specific or more specific disabilities and also there's an indicator in the Child File too about disabilities. So when you link them you have access to more information from the other data sets. [Tammy White] Jamieson I see you asked the question about whether we have file or tabulations to check against. I don't have any published up on the website but Michael I will pump that answer to you because I don't know how much NDACAN has out there that validates what people come up with. Do you guys do that? [Michael Dineen] I'm not sure we'd have to clear that with the Children's Bureau to see if we can do that. [Tammy White] okay [Michael Dineen] because that's giving information out to people [Tammy White] right. Jamieson I would recommend putting that request into NDACAN and then say can forward it to through us and then we can figure out how to answer that for you because if it is just giving you a table to validate some of your stuff then we may have some of that readily available but that will go put that request to the Archive and we can try to funnel it through and see how it gets answered at our end. [Erin McAuley] and so I received a private chat question: will you be providing recommendations on how to better engage our 21-year-old cohort, particularly for those who are no longer in care? [Telisa Burt] I think the Children's Bureau as a whole in NYTD and our work with NYTD and our center for states and all of our technical assistance arms we will be working to help states you know again access resources to best practices that help improve engagement. I know that we do have a lot of work with our young adult reviewers who work with a lot of the state youth councils to help get the word out about NYTD. We talk about a variety of ways to help try to improve that from a youth engagement standpoint and then there are the more scientific ways that are evidenced-based in how to support improved participation rates with long-term studies so will be doing our part in making sure that states have access to those resources. [Erin McAuley] any last burning questions? Okay well then we will wrap up the session for today. I want to thank again to Lisa and Tammy that was absolutely a wonderful presentation and for everyone who came out to participate we are really excited about the session hopefully you guys have been able to get stuff out of it already. Our next presentation is next Wednesday. We'll start a little after 12 again and have the presentation be approximately 30 minutes with ample time for questions and answers so if you come up with any questions between now and then just write them down so you're ready for the chat. I'll be giving the next presentation and it will be about the data structure so were going to be looking at that service file and then the outcomes file cohorts. I'm going to go over how to use the website to get the data and then I'm also going to touch on the available resources at the BCTR in addition to just providing data for people to use for research we also provide a lot of resources that our data users can take advantage of. And then the rest of the presentations will be on Wednesdays through September 19. So thank everyone so much for coming out and we will hopefully see you next week. [Tammy White] thank you everyone I appreciate your time. [Telisa Burt] yes [Voiceover] The national data archive on child abuse and neglect is a project of the Bronfenbrenner Center for Translational Research at Cornell University. Funding for NDACAN is provided by the Children's Bureau.