2018-07-25

Elixir Project Update 7

Welcome back! Overall I had a good week, even though my tasks don't look like it. I probably spent a few extra hours on top of what I had originally planned to working on this list. You can see the repository here, and the server here. Aight, lets get to it.

Last Week:

  • Answer Key - 1 hour
  • Put web code on Github - 15 minutes
  • Setup web server - 45 minutes
  • Research storage options - 1-2 hours
  • Return the Quiz in HTML - 2 hours
  • PDF output - 2 hours
  • Answer Key page or section - 1 hour

The farther off the beaten path the more unknowns, and time sinks I am hitting. I expected this, but it can still be a bit disheartening. This week setting up the web server was that time sink. Most of what I wanted to do was standard, and took me all of 15 minutes. For the non standard part, I sunk three or more hours into trying to figure it out. Finally I compromised, with a partial solution. 

Most tutorials assume you will deploy your web application to a root web directory. So in my case that would have been https://peelle.org/. I already have a website working there so I deployed my Elixir app to https://peelle.org/quiz/ For everything but the static assets like css files and images this worked mostly fine. 

In the router I could set /quiz/ as the root scope, and be done. Pages directed there fine, but static stuff is handled differently. I ended up manually changing some of the paths to just have /quiz/ at the front.

I think I found the correct setting to fix this, and tried multiple ways to set it properly, but was not successful. If I were a younger developer I would probably try and submit a bug report, but I'm an experienced enough programmer, and inexperienced enough Elixir programmer to know that it is much more likely I am doing something wrong.
Sorry didn't have any good code pictures.
So I picked a random one.

Also, I'm a bit too prideful/lazy so I didn't go ask for help like I should have. I made a deal with myself that I will keep trying after I get other tasks done, and if I can't figure it out before the next Elixir meetup I'll ask someone there for help.

As for returning the Quiz in HTML form, it ís not a total bust. I did manage to get it to return data and started figuring out how to play with the data. :-D What I also realized was "Return the Quiz in HTML" was not as atomic as I thought. It has several bits that need to be done to get me there.

Storage options I skimmed some stuff, but I didn't finish it up. I'll do that this week. Everything else I just didn't get to before the week was done.

This Week:
  • Finish storage research.
  • Hook quiz_generator into quiz_website.
    • Decide if this needs to be an OTP app, or just some loaded lib.
    • Determine if the returned information should be persistent, semi persistent, or transient.
  • Return a static page with the Quiz DS.
  • Style page into proper looking quiz, not just a variable dump.
  • Add the Answer DS at the bottom of the webpage.
  • Return a PDF.
  • Add some web tests.
Bonus If I have time:
  • Add authentication.
  • Using storage options, save input lists for a user.
  • Create some pages to manage this.
So this week is about putting the two "apps" together into a working useful web page. Most of my hesitation is more about how to go about that. I think that wrapping my quiz_generator in OTP is probably overkill, but it also looks like fun. :-D 

Also, for storage, I'm trying to decide do I want these quizzes to stay around at all, and if so for how long? Deciding on that could affect if I use ETS or DETS, and so on. 

So another four weeks have passed and I think I'm starting to notice a pattern. My outline, or crystal ball, starts to break down once I get past third week mark. Two weeks back deviated from the plan somewhat, and this last week deviated even more. 

I won't post another massive outline, but I'll probably show a mini road map for the last  portion of the project on the next post. 

2018-07-20

TOCFL Project Weeks 5 & 6

Welcome back!  Long time no see. How ya been?

So over the last two weeks I have kept up with reviewing flash cards but not adding them. :-( Adding has become the bane of this project. My procrastinating, inability to add new cards has gotten so bad I honestly thought about hiring out the job. 

Luckily for me, someone else put them into Memrise. This makes me really happy because I already have a Memrise account, and can just start using the lists immediately, and also because I have used Memrise lists in the past to cram large amounts of words in a short period of time.

Why didn't I check there first you ask? I actually did, but it was a year or two ago, and what they had at the time was incomplete. These new lists look to be complete, and much cleaner than the spread sheet I have been using to create my flash cards.

Aight, so I'm sure you are wondering how the mock test went. Frankly, it went horribly. I walked in thinking I was going to ace it, and I bombed hard. Which is fine. That's the point of taking the mock tests. I needed to know what I was getting into before I showed up for the real McCoy.

For the record I took the Mock B band test, and I barely missed passing the listening and just barely passed the reading for B3. For a few hours after the test I was kicking myself. After doing the whole asses the situation routine, I realized I walked into the test thinking I knew all of the band B vocab, and I only knew a little over half of it. :-O I had unrealistic expectations. 

I had a bunch of other issues that prevented me from performing better, but they are easy to fix. I will test my fixes on the second mock test. These issues are things like the room was freezing, I didn't eat breakfast, only slept 5 hours, had an upset stomach, and I forgot to wear my glasses. 

There were also some legitimate issues. One was, I read a bit slow and had to guess for the last 4 questions in that section. Another was a lot of the questions centered around school and work situations. I work from home, and don't attend school. I will need to practice some of these situations with my tutor so I can get a feel for that.

I blame my failure at listening mostly on the fact that I couldn't focus at all for the first 30 minutes. Although, it wouldn't hurt for me to practice listening to more fast dialogue. The pace wasn't too fast, but they weren't pausing, so anytime I hit a term I didn't know I had to keep on point, listening and hope context would help me out.

So some of my changes going forward are, switching to the Memrise courses, start listening to some Glossika business GSR files, and possibly work through some of the regular glossika GMS files. Overall I am still confident I can pass the C band in Novemeber, but probably only C5 not C6.

I'll post again in another 2 weeks. Wish me luck!

2018-07-18

Elixir Project Update 6

Ok, I am a bit behind. I didn't get most of "last week's" stuff done until Tuesday night. :-( Which means I haven't yet started this weeks either. I got about 1/2 of what I had planned done.

Last Week:

  • Quiz Form - 1 hour
  • Answer Key DS -  1 hour
  • Add optional column support - 1 hour
  • 3 way matching - 30 minutes
  • Put web code on Github - 15 minutes.
  • Setup web server - 1-2 hours?
  • Research storage options - 1-2 hours.

I had a lot of trial and error coding. First was my attempt to see if I could short circuit writing all the bits needed to display a form. I tried using the phx.gen.* stuff, but it didn't end well. In one attempt to generate the quiz form I ended up dumping files that wouldn't compile into the website skeleton. I didn't want to manually rm 10+ files each time I ran that command so I ended up repeatedly regenerating the skeleton. Maybe I should have done that git hub task first.

I used probably a good hour and a half trying various things before moving onto manually writing the code and html templates needed. While doing this I had some fun learning more about eex templates, and Phoenix.HTML.form helpers. I've only had exposure to a few html templating systems, and it is interesting to see how different groups solve this problem.
Ugly version 1

For optional column support I enabled it just for three way matching only, and there is a standard name. It isn't as free form as I had previously envisioned, but for now I think that's for the best.  While I was in there I figured out how to clean up the code a bit, and make it a little more readable. 

While I haven't set up the web server yet, I did get halfway there. I read two tutorials on how to set it up, so I should be able to do it much faster than I previously guessed.  I'll get it done this weekend for sure, along with putting the web code up on github.

For the answer key I am a bit conflicted. I am not sure if I should make it an option, and return a key along with each section, or if I should let someone send in an array of sections, and generate an answer key for the whole thing at once. Also, what kinda key should I return for the matching and three way matching sections?

This week:
  • Answer Key - 1 hour
  • Put web code on Github - 15 minutes
  • Setup web server - 45 minutes
  • Research storage options - 1-2 hours
  • Return the Quiz in HTML - 2 hours
  • Pdf output - 2 hours
  • Answer Key page or section - 1 hour
I foresee more learning pains when I figure out how to hook up the generator app to the framework, as well as working with this form. In a weird way I am looking forward to the frustration. These are things I need to understand well if I want to keep using Elixir for web based tasks. 

PDF output is dependent on there being a library already written, and simple enough for me a new user to understand. So this task will be a little bit research, and hopefully just plugging in some data structures into a set of functions to generate the PDF file. 

Talk at ya next week.


2018-07-15

Elixir Project Update 5

A bit late posting this week. Sorry about that.

So far things are on track. I decided to push back the form one week while I learned more about how Elixir deals with databases, and Phoenix deals with authentication. Here's what I got done last week.


Last Weeks Goals:
  • Phoenix Book Chapter 7 - 1 hour
  • Second Phoenix Book Chapter 6 - 1.5 hours
  • Web app skeleton - 15 minutes
  • Initial quiz form - 1.5 hours
Bonus:
  • Ecto Tutorial - 30 minutes
  • A few one off scripts.1.5 hours
  • Second Phoenix Book Chapter 7 - 1 hour

    I decided to finish up the second book also. It was quite interesting. The last 2 chapters was about tying everything together between the independent game app, and a Phoenix interface.

    I played around with some Project Euler problems. It gave me some practice with guards, recursion, and comprehensions. It was also a fun diversion from other real world stuff.


    This weeks Goals:

    • Quiz Form - 1 hour
    • Answer Key DS -  1 hour
    • Add optional column support - 1 hour
    • 3 way matching - 30 minutes
    • Put web code on Github - 15 minutes.
    • Setup web server - 1-2 hours?
    • Research storage options - 1-2 hours.



    This week is almost over, but I'll leave it's results to the next post. :-D The point in this week is to finish up the quiz_generator, and start figuring out how to fit it together with Phoenix.

    It is also about figuring out how to make it play nice with my current web server. While that isn't strictly programming it usually contains some interesting challenges. One of the cool features I am excited about is the hot patching. I am very interesting to see that in action in a production like environment.

    Ok, I'll have the next post up on Wednesday.  

    2018-07-05

    TOCFL Project Week 3 & 4


    Sorry for the missed week. Like I mentioned on the other project I had a busy work week. I was also a bit lazy. Things are mostly going as scheduled.

    Here is where we basically stand. I have about 3500 vocab terms left to learn by November. That's well within the original guess. I have a Mock tests onsite this weekend.. I will do the B band test. The point in this test is to both get a feel of what it's like to do the real test, to practice. Build some confidence, and to see if there are any glaring problems I have with intermediate level material.

    阿里山

    It kinda seems silly to keep posting every week for this project since it's mostly, study new terms, take a practice test, and repeat. It's not like I'm having a ton of amazing insights, or interesting stuff happening each week. Because  of that, I am going to blog every 2 weeks from now on.

    Like with the Elixir project, I have an updated outline. I'll put it down below. That's all for now, I'll pop back in in 2 weeks with another update, and let ya know how I did on this mock test.


    1. Set Deadline & milestone.
      1. End Date: November 3rd - 22 weeks away. 17 Weeks.
      2. Milestones:
        1. 2018-06-15: Learn ¼ of new vocab and grammar. Pass A band Mock test.
        2. 2018-07-07: Pass B band Mock test. Pass Onsite Test.
        3. 2018-09-12:  Learn ½ of new vocab and grammar.
        4. 2018-10-17: Learn all of new vocab and grammar.
        5. 2018-10-25: Pass Onsite C band Mock Test.
    2. Break down into week by week process.
      1. Take A band Mock Test, and assess. Learn 315 Vocab. Generate new vocab cards from the 高階級 vocabulary list. Find resources for Grammar.
      2. Learn 315 Vocab. Practice new grammar.
      3. Learn 315 Vocab. Take & Pass A band Test with High marks.
      4. Learn 315 Vocab. Practice new grammar. Take online Mock test.
      5. Learn 315 Vocab. Practice new grammar. Review All vocab up to this point.
      6. Learn 250 Vocab. Practice new grammar. Onsite Mock Test.
      7. Learn 250 Vocab. Practice new grammar. Take online Mock C band Test.
      8. Learn 250 Vocab. Practice new grammar. Start using Elixir App. Have tutor do a new assessment to plug weak spots.
      9. Learn 250 Vocab. Practice new grammar. Take it easy.
      10. Learn 250 Vocab. Practice new grammar. Take online Mock C band test.
      11. Learn 315 Vocab. Practice new grammar. Do self generated test.
      12. Learn 250 Vocab. Practice new grammar. Have tutor do a new assessment to plug weak spots.
      13. Learn 250 Vocab. Practice new grammar. Vocab & Grammar ½ way mark. Do self generated test.
      14. Learn 250 Vocab. Practice new grammar. Review Vocab, and take it easy.
      15. Learn 250 Vocab. Practice new grammar. Take online Mock test.
      16. Learn 250 Vocab. Do Onsite C band mock test.
      17. Learn 250 Vocab. Practice new grammar. Work on problems exposed by onsite test.
      18. Learn 250 Vocab. Practice new grammar. Register for the test. Rest.
      19. Review All vocab up to this point. Worst case scenario I should have seen all vocabulary at this point. Take self generated mock test.
      20. Review all grammar. Take online mock test.
      21. Review, practice, repeat.
      22. TEST Week: Review, practice, repeat.

    2018-07-04

    Elixir Project Update 4


    Happy Independence day! I had several late nights at work last week, but overall I did pretty good. I decided to push the web application off until this week, but I got everything else done or almost done before Monday morning.

    Last week's goals:
    • Phoenix Book Chapter 6, & maybe 7 - 1 hour
    • Second Phoenix Book Chapter 5 - 1.5 hours
    • Implement options... or else. :-(  - 2 hours
    • Clean up documentation - 30 minutes
    • Web app skeleton - 15 minutes
    • Add 5 tests - 30 minutes
    While I did most everything, there were most definitely bumps in the road. In both of the books this week I came across code examples that didn't work, and with the time allotted I was not able to work out a solution. I am not saying the books are bad, the main book is still in Beta. There is good news. I found out Sunday that there is a new beta release of the first book. I downloaded it and I will go over it to see if I can resolve my issue.

    While I ran into issues with coding along, I still read both chapters. I just got less out of them. This week I plan to circling back on those chapters. If I can't make any headway in a short time period, then I'll skip ahead and just read the last chapters without following along in the code. Learning the material is my goal, not debugging beta books.
    Random shot from an
    Elixir meetup I went to.

    I got the options implemented, but it took until Tuesday Morning to get them fully implemented. I attempted to use default arguments, but I didn't understand Elixir's implementation that well, and the errors, I understood even less, but I eventually figured it out.

    This week's goals:

    • Phoenix Book Chapter 7 - 1 hour
    • Second Phoenix Book Chapter 6 - 1.5 hours
    • Web app skeleton - 15 minutes
    • Initial quiz form - 1.5 hours
    Since it's a holiday week I am taking it a bit easy, and easing off the Elixir a bit for this week. Also, I am doing extra for the Chinese project this week. I have a Pilot test this weekend which will give me a real feel of what to expect for the end goal. More on that in it's own post.

    I mentioned early on in my updates that the more I worked on this project and learned, the more I was better able to estimate, and clean up requirements. Below is a restructured outline like I had on the first post. 

    I have dropped the Chinese specific stuff because most of what I wanted to do there is actually accomplished by just adding the ability to handle optional columns. 

    I also adjusted estimates, and added a section for items that were taking up my time that I had not factored into the original outline. If this new estimate holds true I should be done with this project two weeks early. 

    Also, the other projects blog will be out soon. I know it's behind. :-( That's it for now, I'll post again next week. 



    • Estimate Time required. - Bottom Up Analysis
      • Total Time: 95 hours. 51 hours + repeating = ~54 hours.
        • Work through Phoenix Books: 10 hours. 4 hours.
        • Quiz generation: 24 hours. 6 hours.
          • Milestone 1: 6 hours
            • Take in a kv list of characters & definitions: 2 hours
            • Generate multiple choice questions based on list: 2 hours
            • Return a quiz data structure: 2 hours
          • Milestone 2: 6 hours
            • Generate a fill in the blank ( character writitng ) section. 2 hours
            • Generate a matching section. 2 hours
            • Add options to limit questions for each section. 2 hours
          • Milestone 3: 6 hours.
            • answer key DS. 2 hours.
            • Add optional column support 2 hour.
            • Generate 3 way match questions. 2 hour.
        • Website: 52 hours. 33 hours.
          • Milestone 1: 13 hours.
            • Site Hello world: 15 Minutes.
            • Quiz generation form. 1 hours.
            • Generated Quiz in HTML format. 4 hours.
            • Add PDF output. 4 hours.
            • Add Key page. 2 hours.
            • Update quiz generation form for question and choice options. 1 hours.
          • Milestone 2: 12 hours.
            • Add login/logout 2 hours.
            • Add functionality to save input list. 2 hours.
            • Add saved input list page. 1 hours.
            • Update form to optionally use saved input lists. 1 hours.
            • Add ability to submit xls, xlsx, and tsv. 4 hours.
            • Add the ability to manually enter word lists. 2 hours.
          • Milestone 3: 8 hours.
            • Add the ability to use multiple word lists on one quiz: 2 hours.
            • Add the ability to tag word lists: 2 hours.
            • Add ability to download xls xlsx and tsv. 2 hours.
            • Update quiz generation form for optional columns. 2 hours.
        • All the tasks that don’t neatly fit into the main categories.
          • Code – 8 hours + repeating.
            • Setup Quiz Github Repo
            • Basic code base maintenance(tests, cleanup, commits, & docs). Repeating 15 min/week.
            • Setup website Github Repo. 30 minutes.
            • Setup website under peelle.org. 5 hours.
            • Misc config/setup( packages, db, etc). 2 hours.
    • Set Deadline & milestone.
      • Deadline. 2018-10-04 ( ~18 weeks ) 2018-08-27 ( ~ 12 weeks ).
      • Milestones:
        • 2018-06-13: Quiz Gen MS 1 done. Phoenix Book Ch 1 – 3 done.
        • 2018-06-27: Quiz Gen MS 2 done.
        • 2018-07-15: Website MS 1. Phoenix Books Final Chapters.
        • 2018-07-29: Quiz Gen MS 3. Misc Tasks.
        • 2018-08-12: Website MS 2.
        • 2018-08-26: Website MS 3. Misc Tasks.
    • Break down into week by week process.
      • Chapter 1 Of Phoenix book. Take in the KV list of Q&A definition. Generate Questions based on this.
      • Chapter 2, and 3 of Phoenix Book. Generate output DS for questions.
      • Chapter 4 , of Phoenx Book. Add option to limit each question. Add FIB section.
      • Chapter 5. Add Matching section.
      • Web app skeleton/Hello world. Quiz Generator Form. Phoenix B1 & B2 Ch 6.
      • Answer Key DS. Add optional column support. 3 way matching. Setup web server, and web github.
      • Returned quiz in HTML. PDF output, Key page, Add options. Phoenix B1 & B2 Ch 7.
      • Add login/logout. Save input list. Saved input list page.
      • Make input lists usable on the quiz page. Add manual input to the quiz. Add alt format support.
      • Multiple wordlists. Tag word lists. Download alt formats.
      • Optional column support. Last of config/cleanup/testing.

    2018-06-26

    Elixir Project Update 3

    Another week done. I did most everything I set out to do, but ran out of time for a few things. Overall, it is getting easier to write Elixir code. This last week was fun. I'm looking forward to this upcoming week's tasks.

    I purposefully delayed the options and the tests till last. For the tests, they are kinda a nice to have that gives me a bit of practice writing tests in this language, but they are not really a high priority. On Sunday when I was going to do the tests, I ended up helping someone move. Probably healthier for me in the long run. :-D

    The code cleanup was kinda fun. I got rid of a module I never used. I also added some functions that made working with the input list struct more manageable. I think the code looks a bit more like how an Elixir programmer would write it now.

    With the options, I was procrastinating. Honestly I think it is because I only have a vague sense of what I want there. What I mean is I can clearly see the 2 types of options I previously specified, but no others. I don't want to lock it down to just those two and realize later that I need to add some other options. So for this week, before I implement options, I am going to give it a good look, and bite the bullet.

    The first book, while less fun this week was very useful. It walked me through how to authenticate, and add users. I may have misunderstood but it looks like he took me through a more hands on way of doing that, rather than using a module. So on one hand I know more about Phoenix, and how to do another part of my project, but on the other hand I plan to look for an easier possibly more standard way.

    The second book was more enjoyable. It gave me a better understanding of OTP's GenServer. In this book he is making a battleship clone, and he is using GenServer for concurrency; one per game. It is neat seeing him tie it all together.

    Last week's goals:
    • Add options for number of choices per question
    • Add options for number of questions per section
    • Phoenix Book Chapter 5, 6, and maybe 7
    • Second Phoenix Book Chapter 4
    • Add Matching Section
    • Set up the Github repo
    • Add 5 tests
    Bonus?:
    • Reformatted and cleaned up code.
    • Made a quick script to help automate a local task
    The github repo is,up. I'll fix the documentation this week. Feel free to check out the code and let me know if I am making any major mistakes. 

    This week's goals:
    • Phoenix Book Chapter 6, & maybe 7 - 1 hour
    • Second Phoenix Book Chapter 5 - 1.5 hours
    • Implement options... or else. :-(  - 2 hours
    • Clean up documentation - 30 minutes
    • Web app skeleton - 15 minutes
    • Add 5 tests - 30 minutes
    Everything I am doing this week is fairly self explanatory. I should be able to knock out this list no problem. /me knocks on wood. 

    Next week's post I'll put up an updated outline. At that point we'll have covered almost 1/4 of the project, and 2 of the original milestones would be reached.