2018-07-25

Elixir Project Update 7

Welcome back! Overall I had a good week, even though my tasks don't look like it. I probably spent a few extra hours on top of what I had originally planned to working on this list. You can see the repository here, and the server here. Aight, lets get to it.

Last Week:

  • Answer Key - 1 hour
  • Put web code on Github - 15 minutes
  • Setup web server - 45 minutes
  • Research storage options - 1-2 hours
  • Return the Quiz in HTML - 2 hours
  • PDF output - 2 hours
  • Answer Key page or section - 1 hour

The farther off the beaten path the more unknowns, and time sinks I am hitting. I expected this, but it can still be a bit disheartening. This week setting up the web server was that time sink. Most of what I wanted to do was standard, and took me all of 15 minutes. For the non standard part, I sunk three or more hours into trying to figure it out. Finally I compromised, with a partial solution. 

Most tutorials assume you will deploy your web application to a root web directory. So in my case that would have been https://peelle.org/. I already have a website working there so I deployed my Elixir app to https://peelle.org/quiz/ For everything but the static assets like css files and images this worked mostly fine. 

In the router I could set /quiz/ as the root scope, and be done. Pages directed there fine, but static stuff is handled differently. I ended up manually changing some of the paths to just have /quiz/ at the front.

I think I found the correct setting to fix this, and tried multiple ways to set it properly, but was not successful. If I were a younger developer I would probably try and submit a bug report, but I'm an experienced enough programmer, and inexperienced enough Elixir programmer to know that it is much more likely I am doing something wrong.
Sorry didn't have any good code pictures.
So I picked a random one.

Also, I'm a bit too prideful/lazy so I didn't go ask for help like I should have. I made a deal with myself that I will keep trying after I get other tasks done, and if I can't figure it out before the next Elixir meetup I'll ask someone there for help.

As for returning the Quiz in HTML form, it ís not a total bust. I did manage to get it to return data and started figuring out how to play with the data. :-D What I also realized was "Return the Quiz in HTML" was not as atomic as I thought. It has several bits that need to be done to get me there.

Storage options I skimmed some stuff, but I didn't finish it up. I'll do that this week. Everything else I just didn't get to before the week was done.

This Week:
  • Finish storage research.
  • Hook quiz_generator into quiz_website.
    • Decide if this needs to be an OTP app, or just some loaded lib.
    • Determine if the returned information should be persistent, semi persistent, or transient.
  • Return a static page with the Quiz DS.
  • Style page into proper looking quiz, not just a variable dump.
  • Add the Answer DS at the bottom of the webpage.
  • Return a PDF.
  • Add some web tests.
Bonus If I have time:
  • Add authentication.
  • Using storage options, save input lists for a user.
  • Create some pages to manage this.
So this week is about putting the two "apps" together into a working useful web page. Most of my hesitation is more about how to go about that. I think that wrapping my quiz_generator in OTP is probably overkill, but it also looks like fun. :-D 

Also, for storage, I'm trying to decide do I want these quizzes to stay around at all, and if so for how long? Deciding on that could affect if I use ETS or DETS, and so on. 

So another four weeks have passed and I think I'm starting to notice a pattern. My outline, or crystal ball, starts to break down once I get past third week mark. Two weeks back deviated from the plan somewhat, and this last week deviated even more. 

I won't post another massive outline, but I'll probably show a mini road map for the last  portion of the project on the next post. 

2018-07-20

TOCFL Project Weeks 5 & 6

Welcome back!  Long time no see. How ya been?

So over the last two weeks I have kept up with reviewing flash cards but not adding them. :-( Adding has become the bane of this project. My procrastinating, inability to add new cards has gotten so bad I honestly thought about hiring out the job. 

Luckily for me, someone else put them into Memrise. This makes me really happy because I already have a Memrise account, and can just start using the lists immediately, and also because I have used Memrise lists in the past to cram large amounts of words in a short period of time.

Why didn't I check there first you ask? I actually did, but it was a year or two ago, and what they had at the time was incomplete. These new lists look to be complete, and much cleaner than the spread sheet I have been using to create my flash cards.

Aight, so I'm sure you are wondering how the mock test went. Frankly, it went horribly. I walked in thinking I was going to ace it, and I bombed hard. Which is fine. That's the point of taking the mock tests. I needed to know what I was getting into before I showed up for the real McCoy.

For the record I took the Mock B band test, and I barely missed passing the listening and just barely passed the reading for B3. For a few hours after the test I was kicking myself. After doing the whole asses the situation routine, I realized I walked into the test thinking I knew all of the band B vocab, and I only knew a little over half of it. :-O I had unrealistic expectations. 

I had a bunch of other issues that prevented me from performing better, but they are easy to fix. I will test my fixes on the second mock test. These issues are things like the room was freezing, I didn't eat breakfast, only slept 5 hours, had an upset stomach, and I forgot to wear my glasses. 

There were also some legitimate issues. One was, I read a bit slow and had to guess for the last 4 questions in that section. Another was a lot of the questions centered around school and work situations. I work from home, and don't attend school. I will need to practice some of these situations with my tutor so I can get a feel for that.

I blame my failure at listening mostly on the fact that I couldn't focus at all for the first 30 minutes. Although, it wouldn't hurt for me to practice listening to more fast dialogue. The pace wasn't too fast, but they weren't pausing, so anytime I hit a term I didn't know I had to keep on point, listening and hope context would help me out.

So some of my changes going forward are, switching to the Memrise courses, start listening to some Glossika business GSR files, and possibly work through some of the regular glossika GMS files. Overall I am still confident I can pass the C band in Novemeber, but probably only C5 not C6.

I'll post again in another 2 weeks. Wish me luck!

2018-07-18

Elixir Project Update 6

Ok, I am a bit behind. I didn't get most of "last week's" stuff done until Tuesday night. :-( Which means I haven't yet started this weeks either. I got about 1/2 of what I had planned done.

Last Week:

  • Quiz Form - 1 hour
  • Answer Key DS -  1 hour
  • Add optional column support - 1 hour
  • 3 way matching - 30 minutes
  • Put web code on Github - 15 minutes.
  • Setup web server - 1-2 hours?
  • Research storage options - 1-2 hours.

I had a lot of trial and error coding. First was my attempt to see if I could short circuit writing all the bits needed to display a form. I tried using the phx.gen.* stuff, but it didn't end well. In one attempt to generate the quiz form I ended up dumping files that wouldn't compile into the website skeleton. I didn't want to manually rm 10+ files each time I ran that command so I ended up repeatedly regenerating the skeleton. Maybe I should have done that git hub task first.

I used probably a good hour and a half trying various things before moving onto manually writing the code and html templates needed. While doing this I had some fun learning more about eex templates, and Phoenix.HTML.form helpers. I've only had exposure to a few html templating systems, and it is interesting to see how different groups solve this problem.
Ugly version 1

For optional column support I enabled it just for three way matching only, and there is a standard name. It isn't as free form as I had previously envisioned, but for now I think that's for the best.  While I was in there I figured out how to clean up the code a bit, and make it a little more readable. 

While I haven't set up the web server yet, I did get halfway there. I read two tutorials on how to set it up, so I should be able to do it much faster than I previously guessed.  I'll get it done this weekend for sure, along with putting the web code up on github.

For the answer key I am a bit conflicted. I am not sure if I should make it an option, and return a key along with each section, or if I should let someone send in an array of sections, and generate an answer key for the whole thing at once. Also, what kinda key should I return for the matching and three way matching sections?

This week:
  • Answer Key - 1 hour
  • Put web code on Github - 15 minutes
  • Setup web server - 45 minutes
  • Research storage options - 1-2 hours
  • Return the Quiz in HTML - 2 hours
  • Pdf output - 2 hours
  • Answer Key page or section - 1 hour
I foresee more learning pains when I figure out how to hook up the generator app to the framework, as well as working with this form. In a weird way I am looking forward to the frustration. These are things I need to understand well if I want to keep using Elixir for web based tasks. 

PDF output is dependent on there being a library already written, and simple enough for me a new user to understand. So this task will be a little bit research, and hopefully just plugging in some data structures into a set of functions to generate the PDF file. 

Talk at ya next week.


2018-07-15

Elixir Project Update 5

A bit late posting this week. Sorry about that.

So far things are on track. I decided to push back the form one week while I learned more about how Elixir deals with databases, and Phoenix deals with authentication. Here's what I got done last week.


Last Weeks Goals:
  • Phoenix Book Chapter 7 - 1 hour
  • Second Phoenix Book Chapter 6 - 1.5 hours
  • Web app skeleton - 15 minutes
  • Initial quiz form - 1.5 hours
Bonus:
  • Ecto Tutorial - 30 minutes
  • A few one off scripts.1.5 hours
  • Second Phoenix Book Chapter 7 - 1 hour

    I decided to finish up the second book also. It was quite interesting. The last 2 chapters was about tying everything together between the independent game app, and a Phoenix interface.

    I played around with some Project Euler problems. It gave me some practice with guards, recursion, and comprehensions. It was also a fun diversion from other real world stuff.


    This weeks Goals:

    • Quiz Form - 1 hour
    • Answer Key DS -  1 hour
    • Add optional column support - 1 hour
    • 3 way matching - 30 minutes
    • Put web code on Github - 15 minutes.
    • Setup web server - 1-2 hours?
    • Research storage options - 1-2 hours.



    This week is almost over, but I'll leave it's results to the next post. :-D The point in this week is to finish up the quiz_generator, and start figuring out how to fit it together with Phoenix.

    It is also about figuring out how to make it play nice with my current web server. While that isn't strictly programming it usually contains some interesting challenges. One of the cool features I am excited about is the hot patching. I am very interesting to see that in action in a production like environment.

    Ok, I'll have the next post up on Wednesday.  

    2018-07-05

    TOCFL Project Week 3 & 4


    Sorry for the missed week. Like I mentioned on the other project I had a busy work week. I was also a bit lazy. Things are mostly going as scheduled.

    Here is where we basically stand. I have about 3500 vocab terms left to learn by November. That's well within the original guess. I have a Mock tests onsite this weekend.. I will do the B band test. The point in this test is to both get a feel of what it's like to do the real test, to practice. Build some confidence, and to see if there are any glaring problems I have with intermediate level material.

    阿里山

    It kinda seems silly to keep posting every week for this project since it's mostly, study new terms, take a practice test, and repeat. It's not like I'm having a ton of amazing insights, or interesting stuff happening each week. Because  of that, I am going to blog every 2 weeks from now on.

    Like with the Elixir project, I have an updated outline. I'll put it down below. That's all for now, I'll pop back in in 2 weeks with another update, and let ya know how I did on this mock test.


    1. Set Deadline & milestone.
      1. End Date: November 3rd - 22 weeks away. 17 Weeks.
      2. Milestones:
        1. 2018-06-15: Learn ¼ of new vocab and grammar. Pass A band Mock test.
        2. 2018-07-07: Pass B band Mock test. Pass Onsite Test.
        3. 2018-09-12:  Learn ½ of new vocab and grammar.
        4. 2018-10-17: Learn all of new vocab and grammar.
        5. 2018-10-25: Pass Onsite C band Mock Test.
    2. Break down into week by week process.
      1. Take A band Mock Test, and assess. Learn 315 Vocab. Generate new vocab cards from the 高階級 vocabulary list. Find resources for Grammar.
      2. Learn 315 Vocab. Practice new grammar.
      3. Learn 315 Vocab. Take & Pass A band Test with High marks.
      4. Learn 315 Vocab. Practice new grammar. Take online Mock test.
      5. Learn 315 Vocab. Practice new grammar. Review All vocab up to this point.
      6. Learn 250 Vocab. Practice new grammar. Onsite Mock Test.
      7. Learn 250 Vocab. Practice new grammar. Take online Mock C band Test.
      8. Learn 250 Vocab. Practice new grammar. Start using Elixir App. Have tutor do a new assessment to plug weak spots.
      9. Learn 250 Vocab. Practice new grammar. Take it easy.
      10. Learn 250 Vocab. Practice new grammar. Take online Mock C band test.
      11. Learn 315 Vocab. Practice new grammar. Do self generated test.
      12. Learn 250 Vocab. Practice new grammar. Have tutor do a new assessment to plug weak spots.
      13. Learn 250 Vocab. Practice new grammar. Vocab & Grammar ½ way mark. Do self generated test.
      14. Learn 250 Vocab. Practice new grammar. Review Vocab, and take it easy.
      15. Learn 250 Vocab. Practice new grammar. Take online Mock test.
      16. Learn 250 Vocab. Do Onsite C band mock test.
      17. Learn 250 Vocab. Practice new grammar. Work on problems exposed by onsite test.
      18. Learn 250 Vocab. Practice new grammar. Register for the test. Rest.
      19. Review All vocab up to this point. Worst case scenario I should have seen all vocabulary at this point. Take self generated mock test.
      20. Review all grammar. Take online mock test.
      21. Review, practice, repeat.
      22. TEST Week: Review, practice, repeat.

    2018-07-04

    Elixir Project Update 4


    Happy Independence day! I had several late nights at work last week, but overall I did pretty good. I decided to push the web application off until this week, but I got everything else done or almost done before Monday morning.

    Last week's goals:
    • Phoenix Book Chapter 6, & maybe 7 - 1 hour
    • Second Phoenix Book Chapter 5 - 1.5 hours
    • Implement options... or else. :-(  - 2 hours
    • Clean up documentation - 30 minutes
    • Web app skeleton - 15 minutes
    • Add 5 tests - 30 minutes
    While I did most everything, there were most definitely bumps in the road. In both of the books this week I came across code examples that didn't work, and with the time allotted I was not able to work out a solution. I am not saying the books are bad, the main book is still in Beta. There is good news. I found out Sunday that there is a new beta release of the first book. I downloaded it and I will go over it to see if I can resolve my issue.

    While I ran into issues with coding along, I still read both chapters. I just got less out of them. This week I plan to circling back on those chapters. If I can't make any headway in a short time period, then I'll skip ahead and just read the last chapters without following along in the code. Learning the material is my goal, not debugging beta books.
    Random shot from an
    Elixir meetup I went to.

    I got the options implemented, but it took until Tuesday Morning to get them fully implemented. I attempted to use default arguments, but I didn't understand Elixir's implementation that well, and the errors, I understood even less, but I eventually figured it out.

    This week's goals:

    • Phoenix Book Chapter 7 - 1 hour
    • Second Phoenix Book Chapter 6 - 1.5 hours
    • Web app skeleton - 15 minutes
    • Initial quiz form - 1.5 hours
    Since it's a holiday week I am taking it a bit easy, and easing off the Elixir a bit for this week. Also, I am doing extra for the Chinese project this week. I have a Pilot test this weekend which will give me a real feel of what to expect for the end goal. More on that in it's own post.

    I mentioned early on in my updates that the more I worked on this project and learned, the more I was better able to estimate, and clean up requirements. Below is a restructured outline like I had on the first post. 

    I have dropped the Chinese specific stuff because most of what I wanted to do there is actually accomplished by just adding the ability to handle optional columns. 

    I also adjusted estimates, and added a section for items that were taking up my time that I had not factored into the original outline. If this new estimate holds true I should be done with this project two weeks early. 

    Also, the other projects blog will be out soon. I know it's behind. :-( That's it for now, I'll post again next week. 



    • Estimate Time required. - Bottom Up Analysis
      • Total Time: 95 hours. 51 hours + repeating = ~54 hours.
        • Work through Phoenix Books: 10 hours. 4 hours.
        • Quiz generation: 24 hours. 6 hours.
          • Milestone 1: 6 hours
            • Take in a kv list of characters & definitions: 2 hours
            • Generate multiple choice questions based on list: 2 hours
            • Return a quiz data structure: 2 hours
          • Milestone 2: 6 hours
            • Generate a fill in the blank ( character writitng ) section. 2 hours
            • Generate a matching section. 2 hours
            • Add options to limit questions for each section. 2 hours
          • Milestone 3: 6 hours.
            • answer key DS. 2 hours.
            • Add optional column support 2 hour.
            • Generate 3 way match questions. 2 hour.
        • Website: 52 hours. 33 hours.
          • Milestone 1: 13 hours.
            • Site Hello world: 15 Minutes.
            • Quiz generation form. 1 hours.
            • Generated Quiz in HTML format. 4 hours.
            • Add PDF output. 4 hours.
            • Add Key page. 2 hours.
            • Update quiz generation form for question and choice options. 1 hours.
          • Milestone 2: 12 hours.
            • Add login/logout 2 hours.
            • Add functionality to save input list. 2 hours.
            • Add saved input list page. 1 hours.
            • Update form to optionally use saved input lists. 1 hours.
            • Add ability to submit xls, xlsx, and tsv. 4 hours.
            • Add the ability to manually enter word lists. 2 hours.
          • Milestone 3: 8 hours.
            • Add the ability to use multiple word lists on one quiz: 2 hours.
            • Add the ability to tag word lists: 2 hours.
            • Add ability to download xls xlsx and tsv. 2 hours.
            • Update quiz generation form for optional columns. 2 hours.
        • All the tasks that don’t neatly fit into the main categories.
          • Code – 8 hours + repeating.
            • Setup Quiz Github Repo
            • Basic code base maintenance(tests, cleanup, commits, & docs). Repeating 15 min/week.
            • Setup website Github Repo. 30 minutes.
            • Setup website under peelle.org. 5 hours.
            • Misc config/setup( packages, db, etc). 2 hours.
    • Set Deadline & milestone.
      • Deadline. 2018-10-04 ( ~18 weeks ) 2018-08-27 ( ~ 12 weeks ).
      • Milestones:
        • 2018-06-13: Quiz Gen MS 1 done. Phoenix Book Ch 1 – 3 done.
        • 2018-06-27: Quiz Gen MS 2 done.
        • 2018-07-15: Website MS 1. Phoenix Books Final Chapters.
        • 2018-07-29: Quiz Gen MS 3. Misc Tasks.
        • 2018-08-12: Website MS 2.
        • 2018-08-26: Website MS 3. Misc Tasks.
    • Break down into week by week process.
      • Chapter 1 Of Phoenix book. Take in the KV list of Q&A definition. Generate Questions based on this.
      • Chapter 2, and 3 of Phoenix Book. Generate output DS for questions.
      • Chapter 4 , of Phoenx Book. Add option to limit each question. Add FIB section.
      • Chapter 5. Add Matching section.
      • Web app skeleton/Hello world. Quiz Generator Form. Phoenix B1 & B2 Ch 6.
      • Answer Key DS. Add optional column support. 3 way matching. Setup web server, and web github.
      • Returned quiz in HTML. PDF output, Key page, Add options. Phoenix B1 & B2 Ch 7.
      • Add login/logout. Save input list. Saved input list page.
      • Make input lists usable on the quiz page. Add manual input to the quiz. Add alt format support.
      • Multiple wordlists. Tag word lists. Download alt formats.
      • Optional column support. Last of config/cleanup/testing.

    2018-06-26

    Elixir Project Update 3

    Another week done. I did most everything I set out to do, but ran out of time for a few things. Overall, it is getting easier to write Elixir code. This last week was fun. I'm looking forward to this upcoming week's tasks.

    I purposefully delayed the options and the tests till last. For the tests, they are kinda a nice to have that gives me a bit of practice writing tests in this language, but they are not really a high priority. On Sunday when I was going to do the tests, I ended up helping someone move. Probably healthier for me in the long run. :-D

    The code cleanup was kinda fun. I got rid of a module I never used. I also added some functions that made working with the input list struct more manageable. I think the code looks a bit more like how an Elixir programmer would write it now.

    With the options, I was procrastinating. Honestly I think it is because I only have a vague sense of what I want there. What I mean is I can clearly see the 2 types of options I previously specified, but no others. I don't want to lock it down to just those two and realize later that I need to add some other options. So for this week, before I implement options, I am going to give it a good look, and bite the bullet.

    The first book, while less fun this week was very useful. It walked me through how to authenticate, and add users. I may have misunderstood but it looks like he took me through a more hands on way of doing that, rather than using a module. So on one hand I know more about Phoenix, and how to do another part of my project, but on the other hand I plan to look for an easier possibly more standard way.

    The second book was more enjoyable. It gave me a better understanding of OTP's GenServer. In this book he is making a battleship clone, and he is using GenServer for concurrency; one per game. It is neat seeing him tie it all together.

    Last week's goals:
    • Add options for number of choices per question
    • Add options for number of questions per section
    • Phoenix Book Chapter 5, 6, and maybe 7
    • Second Phoenix Book Chapter 4
    • Add Matching Section
    • Set up the Github repo
    • Add 5 tests
    Bonus?:
    • Reformatted and cleaned up code.
    • Made a quick script to help automate a local task
    The github repo is,up. I'll fix the documentation this week. Feel free to check out the code and let me know if I am making any major mistakes. 

    This week's goals:
    • Phoenix Book Chapter 6, & maybe 7 - 1 hour
    • Second Phoenix Book Chapter 5 - 1.5 hours
    • Implement options... or else. :-(  - 2 hours
    • Clean up documentation - 30 minutes
    • Web app skeleton - 15 minutes
    • Add 5 tests - 30 minutes
    Everything I am doing this week is fairly self explanatory. I should be able to knock out this list no problem. /me knocks on wood. 

    Next week's post I'll put up an updated outline. At that point we'll have covered almost 1/4 of the project, and 2 of the original milestones would be reached. 

    2018-06-21

    TOCFL Project Week 2

    Week 2 down. So far things are going ok, but not as good as I would like. As I mentioned on the other project, I had to ease off for a few days this last week. Good news so far this week everything is back to normal.

    Sometimes my reviewing is outpacing the speed flashcards are added to the deck. While I did miss a few days, I did not consistently add cards, but rather did a whole big lump at the end of the week.  Between that and missing a few days, I think I only learned around 200 new terms this week.

    Also, I should have done the mock test, but I left it to the weekend, and that was when I ended up taking time to recover. I already have a mock test planned for this next week, so I won't double test.

    Last Weeks Goals:
    • Learn 315 new terms.
    • Keep processing the 高階級 word list. I need to look over at least 630 terms to keep pace.
    • Practice with my tutor.
    • Take a mock test.
    Because of the previously mentioned spacing issues I plan to just buckle down and finish off the lists.

    There is another TOCFL test that just opened up. It is an adaptive scoring test. It is interesting in that you don't pick a band but rather based on the questions you answer your band and level will be determined. The way I understand it, one way they do this is the questions change based on your answers. Some of the information on their website makes it sound like I can't register for the test unless I'm part of an org or university. I'm going to try anyways. :-D
    Food picture to ease the text.

    This Weeks Goals
    • Learn 315 new terms.
    • Finish the 高階級 word list.
    • Practice with my tutor.
    • Take a mock test.
    • See If I can register for the adaptive test.
    Every week I am aiming for 14 review sessions. Two per day. I am fine with more, especially during my commutes to my tutors place.

    Flash Card Stats:
    • 9 Review Sessions
    • 457 reviewed @ avg. 51/per session.
    • 74.5% answered correctly.
    So some reoccurring issues have caused me to start looking for solutions. One is I have a very mainland centered vocabulary. This shouldn't be a problem, but living in Taiwan, conversing with Taiwanese people, and preparing for a Taiwan made test is repeatedly bringing this to light. Here's a short video showing some ways the two differ. I'm at the point where it's not just a few words, but 10's of words. 

    It just means I have to learn an extra term, and be more situational aware, but to be more aware I have to know that the problem exists. My tutor is more adept at dealing with this problem than other tutors I've had, but there are still plenty of mainland specific terms, or mainland specific usages that she doesn't know. When this happens one of a few frustrating situations occur. 

    First, she tells me I am using something wrong. Neither of us know that I just can't use it in Taiwan. Because I still have friends from mainland, and absorb various mainland media, the problem is brought to my attention days, or weeks after I have the tutor and I have discussed it. In that time it's not productive to bring that point back up in the next session. Also, I may have put some effort into removing that speech pattern, or vocab from my day to day speaking.

    Another is when I know I am using the term right, and we end up wasting time finding out it's just not right for Taiwan. I sometimes prefer this situation because of the immediate feedback, but dislike it because I fear that it might make my tutor feel inadequate or lose face.

    Sometimes these issues are being prompted by my use of Pleco. The free dictionary seems to be mainland centered. So I will see an example sentence, or term in the dictionary app, that isn't normal for Taiwan. I am looking into purchasing a more comprehensive dictionary in the hopes that it will have more Taiwan related terms, and info.

    Just to be clear, knowing all the differences between mainland Mandarin and Taiwan Mandarin is not really in my tutors job description. I'm happy when she does know, but I don't expect it from her.

    Alight that's enough rambling for me. Post at ya next week!

    2018-06-19

    Elixir Project Update 2


    Week 2 is done. I had a few bumps in the road, but overall I did alright. I did not accomplish everything, but I made progress. I have buffers built into the schedule so I am not too worried. I just wish my first missed goal would have been a bit farther along than week 2.

    There was 2 main excuses, and a minor one that tripped me up. First, was a work issue, and the second was I over extended myself. The minor bit was this weekend was a local holiday, and I didn't account for holiday activities.

    I work a weird split shift for work, where I work half my shift in the a.m., and half my shift just before bed. That evening shift is aligned with the main office's morning. I usually have meetings, and interact with coworkers at that time. Probably once every month or two, I end up having meetings run long. Which causes the work evening that usually ends at 11 p.m., to go over to 1 a.m., or 2 a.m.  This last week happened to have one of those days.


    Most weekdays I sleep from roughly 12 a.m. to 8 a.m. Fitbit thinks I sleep around 7 hours inside that period. Usually when those meetings run long I sleep in the following morning, but being on this new schedule I figured I could find time for a couple hour nap later, which didn't happen. Combine that with I am trying to exercise every day, and I essentially ran myself down.

    So, to prevent future issues with those two main problems, I plan to do two things. First I will give myself permission to adjust my schedule when that sort of thing happens, and suspend exercise for the following day. Second I'm going look at upcoming holidays and incorporate them into the schedule.

    Previous week's goals:
    • Phoenix Book Ch 4
    • Second Phoenix Book Ch 3 and maybe 4
    • Add a Fill in the Blank Module.
    • Add options for number of choices per question.
    • Add option for number of questions per section.
    So some observations for this last week. Um first, the FIB section was cake. I should have done it first. It was an easier version of the multiple choice. I was able to add 3 lines of code and have it return the list of questions, and for fun/consistency a string of underline characters for the answer side. Adding the code took all of a few minutes, and testing a few more.

    This fourth chapter in the main Phoenix book took me through connecting a DBMS up to the framework, and making a schema file for it. I was surprised that PostgreSQL was the default choice. It's been probably 10 years since I done anything more than maintain a PostgreSQL server. At that time I worked on a GIS project that needed it. That server still runs, and I "maintain it" but I haven't had to actually manually query any tables, add users, or add any data since the project.  Other than the overlap with other SQL DBMSs I have forgotten most of the practical knowledge I gained using it. :-( So I am kinda cool with the refresher, but there's a good chance I'll use MariaDB when I need a database for the actual project.


    I find the second Phoenix book quite interesting. In this weeks chapter he had us build a state machine for the program. I hadn't done one of those since college, and I really like how it looked in Elixir code. Also, I only did one chapter because this book's chapters are thicker than, some of the other Elixir books I have read. 

    Lastly I think I figured out part of why I was having issues with that data transformation. I researched some, and while I knew that a struct was a map,I didn't grasp all the differences that come along with it being a struct on my first pass through the material.

    This week's goals:
    • Add options for number of choices per question - 1.5 hour
    • Add options for number of questions per section - .5 hour
    • Phoenix Book Chapter 5, 6, and maybe 7 - 1 hour
    • Second Phoenix Book Chapter 4 - 2 hours
    • Add Matching Section - 1 hour
    • Set up the Github repo - 15 minutes
    • Add 5 tests - 30 minutes
    Based on what I figured out about the fill in the blank section, I believe that the matching section should be fairly easy, and have adjusted my estimate of that section to 0.5 - 1 hour.  I also think figuring out the way I want to handle options, and implementing it the first time will be the time sink. So that's why one options task is 1/3 of the other's estimate.

    Also, I think the reason I crammed the last 3 chapters of the main phoenix book onto this week was previously because I wanted to know everything I needed to start on the web stuff the following week. Happily the first few chapters have already covered what I need for the following two weeks. If I get overwhelmed, I am fine with only doing one chapter per week for the next three weeks. 

    What I knew in the back of my mind but didn't admit to myself was setting up the Github repo, installing Elixir to server, setting up the web server to work with a phoenix instance, code cleanup, testing, and so on, all that stuff shouldn't be tacked on at the end, and it will take up some time. So I'm interspersing those tasks into the list, and will do them as I go. 

    Until next week!

    2018-06-13

    TOCFL Project Week 1


    Things are going good on this project also! Hooray! This one has less 1 and done tasks, and just a few same tasks repeated.

    Things accomplished:
    • All new+reviewed vocab
    • Registered for the July pilot test.

    The biggest obstacle right now is keeping myself going through those word lists making flash cards. So far I am only a part the way through 高階級, but I guessed fairly accurately, and I'm averaging around 50% of the terms I know.

    For this upcoming week I need to:
    • Learn 315 new terms.
    • Keep processing the 高階級 word list. I need to look over at least 630 terms to keep pace.
    • Practice with my tutor.
    • Take a mock test.
    I know the mock test wasn't scheduled until week 3. The Rapid Learner course made some good points about testing soon, and often. Also they gave some advice on how to generate my own practice tests. Which is good because it looks like there is only 1 mock test provided per band. Then second reason I plan to take it is I really should know the format, and question types. Taking one is a great way to learn it.

    The practice sessions with my tutor have morphed. Now we spend more time reviewing grammar I came across in studying for this test, and conversation practice. 
    Saw this at a one of the DMV type government buildings
    we needed to visit. So many different titles!

    This first week overall was fairly light, and defiantly much lighter than I anticipate the upcoming weeks to be. I had briefly mentioned in the other post that the 進階級 list has terms that I already had been exposed to. So... I had a massive review on day 1, and on the following day I was hit with a smaller second wave of all the cards I marked as forgotten on day 1. Then on day 6 I was again hit with a big review chunk. The good news was once I got this small tsunami out of the way I didn't have but a few new cards the rest of the week. 

    Flash Card Stats
    • 16 review sessions
    • 1273 cards reviewed @ avg. 42/session
    • 81% answered correctly.
    Again I expect these numbers to change. The more new cards the more reviews I will end up doing, and the lower my correct answer % will be. Like I said in the elixir post, I will try to post on the weekend but if not I will plan to post on Wednesday. 

    2018-06-12

    Elixir Project Update 1



    First week down, only 17 to go. Huzzah! Also, good news! I did all the assigned tasks!

    Some useful news I learned enough, and thought enough about the project in general that I will need to adjust the plan.

    Like I had assumed my estimates were off. This week worked out well enough. My estimate for parsing in some data ended up taking only 1/2 an hour, but generating the questions took back an extra hour. I think I found a weak point with my understanding of the language. I had trouble manipulating the data. Specifically dealing with nested structures of different types. I think this is an issue that will go away as I get more familiar with Elixir.

    Sorry, I accidentally, did 2 extra chapters in the Phoenix book. For some reason when wrote out my weekly goals I combined the first two weeks. The good news is I can now much better estimate some of the website parts, and I can take my time with the last chapters which might be more difficult, or thick.

    I do have a change I plan to implement. While I was ok with the 5 hours originally, I don't like how grouped they are. I mentioned before it is best for programming to have at least a two hour block, of time to work on things like this. I worry if I only practice two days in a row out of every 7 that the stuff won't stick as well. Funnily enough, week 3 the Rapid Learner course basically said as much. So I am planning on adding an hour on Saturday & Sunday each.

    During this time I am not going to pile on more of the project. I have 2 plans for this time. First while I had some free time last week I skimmed another book on Phoenix. I had previously shelved it thinking it was too advanced for me, but after skimming it I realized the author is taking a very similar approach to the book's app as I am to mine. It is only 7 chapters, and I have already read the first two this weekend. I'll finish it out a chapter a week using this new time.

    Second once the book is complete, I will have some fun with Elixir. I will use it to write smaller scripts. Anything from rewriting a bash script into Elixir, all the way to doing some sort of problem set like Exorcism, or Project Euler.

    Alright on to some hard numbers.

    Completed last week:
    • Phoenix Book Ch 1 - 3
    • Basic Quiz program.
      • Takes in a list of questions and answers.
      • generates a multiple choice question based on each.
    • Second Phoenix Book Ch 1 - 2
    Upcoming This week
    • Phoenix Book Ch 4
    • Second Phoenix Book Ch 3 and maybe 4
    • Add a Fill in the Blank Module.
    • Add options for number of choices per question.
    • Add option for number of questions per section.

    I thought about reposting an updated version of last week's outline, but that seemed excessive. So, instead here are some select changes.

    The first change is based on what I have learned so far with the Phoenix books. One whole milestone shrunk by 7 hours. I am hoping the more I learn the more this trend continues.
      • Website: Milestone 1: 9 hours 2 hours
        • Site Hello world: 1 hour 5 minutes
        • Quiz generation form: 4 hours 30 minutes
        • Generated Quiz in HTML format: 4 hours 1 hour

    Next I decided that the input/output formatting probably should be it's own module, and not so tightly coupled with the website. My reason for this is that I want the quiz program that I am working on to be able to be ran as it's own stand alone application. This broken out module can do the parsing. So several of the website features/milestones will shrink and be remade into this module. It should look something like this.
    • Quiz Data Parser
      • MS 1: 2 hours
        • Accept a TSV: 30 minutes
        • Output a TSV: 30 minutes
        • Output a PDF: 1 hour
      • MS 2: 2 hours.
        • If module already exists XLS in/out support. 1 hour.
        • XLSX in/out support. 1 hour.
    As I was coding my brain finally showed up and said "Hey, you know your just coding a generic Quiz website, right? All this pinyin, and tone, and auto translate definition stuff shouldn't be the default." Which prompted me of thinking about how I could correctly accommodate these features. I immediately thought of the age old solution of  setting a quiz type, or template. That allows for future templates of other types. Which means I need to add a few features. I am going to wait a week and let that stew in my brain before I put it to schedule.

    Anywho that's all for now. In theory I have the most free time on the weekends, but the weekend is also the most likely time for unplanned activities. So... I'll plan to post on the weekend but if I miss the weekend I will post on Tuesday.

    2018-06-04

    Mandarin TOCFL Project


    Ok, now to outline the second project. In a lot of ways this project is completely different than the Elixir one. This one is a lot about memorization, and prepping for one specific metric. Pass a test.

    Vocab


    Right now setting up the vocab is the bane of my existence. Making flash cards, via anki, pleco, and on paper has always been a pain. Using flashcards on the other hand is quite enjoyable. I am still making the cards, as I look through these word lists which means I am not 100% done estimating things.

    Some fun people we met eating
    in Xiamen.
    What I do have is a hard maximum, and a likely result. Based on these numbers, I decided on 45 new terms a day. I did this for several reasons I'll go over below. The worst case scenario is 18 weeks. This assumes I don't know a single word on the last 2 lists. I guesstimated the most likely scenario is 11 weeks. This assumes I know about 50% of the 高階級 list, and 30% of  the 流利級 list. Of course the extremely unlikely case that I know both of those lists, then I just have 1 week of flashcards. :-D That's a lot of words per day, but not unheard of. I've done 50/day in the past for 1-3 months at a time.

    There may be a small misunderstanding that's probably happening. Memorize can be a vague nebulous term. I am not trying to memorize these words front to back, every nuance perfectly preserved in my brain. The vast majority of these words I will only need to know them in a passive context for the test. The test is a reading/listening test. When I see, or hear it in a sentence do I know what it means. That is much easier than having to recall what it's major radical, know it's exact tones, how to write it, or to remember it out of context. These are all things I will eventually memorize, but are not top priority for the goal.

    So why did I pick 45 new terms/words per day? First i want to be done with the vocabulary at least a few weeks before the test. This will give me extra time to review and work on problem vocab. Second, once I have worked through the last two word lists, and have a hard number, if it's shorter than 18 weeks, I plan to slow down my new terms per week to match that 18 week goal. Next if 45 becomes too many per day and I start to struggle, I can worst case scenario drop it to 35/per day or less depending how far along I get before that happens, and still learn all the words before the test.

    Here is what I currently know about the vocabulary.
    • 準備一級:I know all 148 terms.
    • 準備二級:I know all 172 terms.
    • 入門級: I know all 187 terms. 
    • 基礎級:497 total: 9 I didn’t know. 1.8% 
    • 進階級: 1492 total: 272 I didn’t know. 18% 
    • 高階級: 2491 total: ?? 
    • 流利級: 2987 total: ??
    I had some interesting observations while going through the list. First, a small portion of those words I didn't recognize I actually knew and had learned through conversation. Another nice realization was that about 1/2 of the 進階級 words I didn't know were words I have learned in the past, so reactivating them should be easier than learning them from scratch.

    Grammar


    So I've focused a lot on the vocabulary, but grammar, and the test itself is important also right? I agree. For grammar I have a few methods of attack. First is any word in the vocab list that's basically not a noun, verb, adverb, or adjective is usually a grammar point. Also, I plan to take apart the mock tests each time I take them, and work with my tutor to locate grammar points I failed and incorporate them into my study.

    Most of my grammar practice will be incorporated into my preexisting tutoring sessions. In homework for these sessions I make sentences. I spoke with my tutor and we plan to have this be about me attempting to use the grammar, and her giving me feed back from here on out. Also our conversation practice is a great place for me to insert and use new grammar. She is good about giving me quick feedback on my spoken grammar, as well as appropriate vocab usage for what I am trying to say. Another thing we work on is my reading comprehension. I will read a passage, and she will quiz me on parts to make sure I understood it.

    Testing

    Something I plan to do at least twice a month is take their online mock tests. As I mentioned in the previous section it should give me a good idea of how to plug holes in my understanding of the grammar, but there's more.

    While I plan to do the mock tests, there's a chance for me to take one real test, and two onsite pilot tests before November. I plan to register and try this. For the real test I currently plan to try for the B band test and see how I fair. That test is at the end of July. It's called computerized adaptive on the schedule, and I am not sure what that means or how it differs from the November test yet.

    Time Constraints

    This goal is more important to me than the Elixir one. I already have more invested in it, and I plan to invest more into it. With Elixir I set aside 5 hours per week, for study. For this project I have set aside 12 scheduled hours, but will probably up it to 15 once I see how things go. One thing most people constantly underestimate, myself included is SRS reviews. They can stack up fast if you miss a few days, or just by learning a lot consistently.  If this gets out of hand I plan to add another 1/2 hour of review in the middle of my weekdays. This should space it out, and alleviate some of the pressure.

    Ok I had alot more to say on this one than the last, but finally here is the outline I made for this
    project. Like the other project I'll put up an update once a week, showing how my progress goes. 
    1. Define broad Goal - Learn Advanced Mandarin
      1. Specific - I want to pass the TOCFL C band.
      2. Why Am I learning this.
        • I enjoy communicating in the language.
        • I want using the language to feel like less of an effort.
        • Passing a high level test is something my wife/parents/friends can brag about/be proud of.
        • Passing the test also gives me an objective measure I can use with prospective tutors, employers, and people who aren’t familiar with language learning.
        • I want to feel like I’ve mastered it even if there is always more to learn.
    1. Select Learning Materials
      • There are mock test books.
      • I have 2 unrelated advanced mandarin books.
      • I have a word list for pertinent vocab, but not grammar.
    1. Estimate Time required.
        1. November Test.
          1. Need to know 8000 Vocab by then.
          2. Need to have a firm grip of the grammar.
    2. Set Deadline & milestone.
      1. End Date: November 3rd - 22 weeks away.
      2. Milestones:
        1. 2018-06-15: Pass A band Mock test with high marks.
        2. 2018-07-04: Learn ¼ of new vocab and grammar.
        3. 2018-07-07: Take the onsite pilot test.
        4. 2018-07-15: Pass B band Mock test.
        5. 2017-07-22: Learn ½ of new vocab and grammar.
        6. 2018-07-28: Take B band formal test.
        7. 2018-08-15: Pass B band mock test with high Marks.
        8. 2018-08-28: Learn ¾ of new vocab and grammar.
        9. 2018-9-18: Take C band Mock test.
        10. 2018-10-01: Learn all of new vocab and grammar.
        11. 2018-10-06: Take last onsite pilot test.
        12. 2018-10-17: Pass C band Mock test.
        13. 2018-10-25: Pass C band Mock test with high marks.
    3. Break down into week by week process.
      1. Take A band Mock Test, and assess. Learn 315 Vocab. Generate new vocab cards from the 高階級 vocabulary list. Find resources for Grammar.
      2. Learn 315 Vocab. Practice new grammar. Generate new vocab cards from the 高階級 vocabulary list.
      3. Learn 315 Vocab. Take & Pass A band Mock Test with High marks. Generate new vocab cards from the 流利級 vocabulary list.
      4. Learn 315 Vocab. Practice new grammar. Generate new vocab cards from the 流利級 vocabulary list.
      5. Learn 315 Vocab. Practice new grammar. Review All vocab up to this point. Take onsite pilot test.
      6. Learn 315 Vocab. Practice new grammar. Generate new vocab cards from the 流利級 vocabulary list.
      7. Learn 315 Vocab. Practice new grammar. Pass B Band Test.
      8. Learn 315 Vocab. Practice new grammar. Start using Elixir App as Vocab Quiz Tool. Take onsite formal test for band B.
      9. Learn 315 Vocab. Practice new grammar.
      10. Learn 315 Vocab. Practice new grammar. Review all vocab up to this point. Should be about ½ the new vocabulary.
      11. Learn 315 Vocab. Practice new grammar. Best guess scenario. I should finish all vocab at this point.
      12. Learn 315 Vocab. Practice new grammar. Take band C mock test.
      13. Learn 315 Vocab. Practice new grammar.
      14. Learn 315 Vocab. Practice new grammar.
      15. Learn 315 Vocab. Practice new grammar.
      16. Learn 315 Vocab. Practice new grammar. Review All vocab up to this point. Should be about ¾ the new vocabulary.
      17. Learn 315 Vocab. Practice new grammar. Take C band Mock Test.
      18. Learn 315 Vocab. Practice new grammar. Pass C band test. Take onsite pilot test. Register for the November test.
      19. Review All vocab up to this point. Worst case scenario I should finish all vocab at this point.
      20. Review. Practice, Mock test.
      21. TEST Week: Review All vocab up to this point. Review all grammar up to this point. Retake Mock Test.
    4. Review & Update.